subject:"Re\: Observations of an Internet Middleman \(Level3\)"

On Friday, May 16, 2014 08:47:53 PM Blake Hudson wrote:

 How residential ISPs recoup costs (or simply increase
 revenue/profit) is another question entirely. I think
 the most insightful comment in this discussion was made
 by Mr. Rick Astley (I assume a pseudonym), when he
 states that ISPs have several options to increase
 revenue A) Increase price of their product, B) Implement
 usage restrictions, or C) Charge someone else/Make
 someone else your customer. I think he successfully
 argues that option C may be the best. As we've seen, the
 wireless market in the US went for option B. We've yet
 to see where the wireline market will go.

Some of the operators, here in Africa, who are venturing 
into FTTH, are continuing on with their data caps.

I suppose if you're primarily a mobile carrier who uses data 
caps to charge for access (and makes lots of money in the 
process for, pretty much, selling nothing), the model 
becomes attractive regardless of the medium.

New providers who are not encumbered by this type of 
thinking, obviously, have a more flexible and forward-
thinking business model.

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Friday, May 16, 2014 08:52:31 PM Christopher Morrow 
wrote:

 is 'symmetric traffic ratios' even relevant though?
 Peering is about offsetting costs, right? it might not
 be important that the ratio be 1:1 or 2:1... or even
 10:1, if it's going to cost you 20x to get the traffic
 over longer/transit/etc paths... or if you have to build
 into some horrific location(s) to access the content in
 question.
 
 Harping on symmetric ratios seems very 1990... and not
 particularly germaine to the conversation at hand.

Agree.

We don't have a ratio requirement, for example. We have a 
if it makes sense requirement.

I'm forced to peer with certain African providers in London 
and Amsterdam because they don't want to peer in Africa, 
where we are literally are an x-connect away from each 
other. And the reasons are not even because either of us is 
larger or smaller than the other... it's just legacy 
thinking and we're the new guy that has grown rapidly.

Now we both have to pay for traffic to get sent to Europe 
and back. How nice...

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Friday, May 16, 2014 09:11:56 PM Blake Hudson wrote:

 But hey, why peer at little or no cost if they
 can instead hold out and possibly peer at a negative
 cost?

Because they hope that, one day, you'll cave and become a 
customer. Isn't that more prestigious :-)?

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Friday, May 16, 2014 09:44:55 PM Scott Helms wrote:

  I don't think that anyone disputes that when you improve
 the upstream you do get an uptick in usage in that
 direction.  What I take issue with is the notion that
 the upstream is anything like downstream even when the
 capacity is there.

Certainly not - what I'm saying is that there can be a lot 
more upstream utilization than we are typically seeing 
today, if that stopper is unblocked.

We can, then, take it from there...

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Observations of an Internet Middleman (Level3) (was: RIP

2014-05-16 Thread Jean-Francois Mezei

On 14-05-15 16:17, Keenan Tims wrote:

 As primarily an eyeball network with a token (8000 quoted) number of transit 
 customers it does not seem reasonable for them to expect balanced ratios on 
 peering links. 


Pardon my ignorance here, but isn't there a massive difference between
settlement-free peering between large transit providers at the core
which happens with balanced traffic,

 and some free peering at local exchanges at the edge where there is no
expectation of balanced traffic, just an oppportunity to exchange
traffic without using transit capacity. (isn't that how CDN nodes in a
exchange works ? Lets ISPs connect to it and bypass transit links to
save money ?

Seems to me like the word peering shouldn't have been used to denote
relationships at the core between the big guys if it is also used at the
edge for a fairly different purpose.

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Thursday, May 15, 2014 09:05:57 PM Joe Greco wrote:

 Hi I'm an Internet company.  I don't actually know what
 the next big thing next year will be but I promise that
 I won't host it on my network and cause our traffic to
 become lopsided.

You mean like almost every other mobile carrier the world 
over, and their data-capped services?

Want to guess how many mobile carrier executives converge 
around a table on a daily basis to discuss how to stem 
growth in demand for traffic by their customers?

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality (was: Wow its been quiet here...

2014-05-16 Thread Rick Astley

What you're missing is that the transit provider is
selling full routes.  The access network is selling
paid peering, which is a tiny fraction of the routes.

Considering they charge on a $per/mb basis I don't think its just routes
they are selling. It looks a lot like they are selling bits. From the
perspective of a content provider it looks like they swivel chair most
those bits to access networks for delivery. That tiny fraction of routes
on the access networks make up most delivered content. In total network
size the access networks are larger although less spread out globally.
Being globally connected is useful but it doesn't make a legitimate case
for having exclusive rights to charge for content being delivered in North
America. If you are planning to serve large scale data over oceanic fiber
it's a strong selling point but that's not the case here. If instead of
$per/mb traffic delivery you want to get into arbitrary justifications
access networks have more directly assigned IP addresses than transit
networks. I'm not making the case that a middle man should never be used,
I'm making the case that they shouldn't be used where there isn't a
requirement for one. Bypassing the middleman is generally better for
everyone but the middle man.


So, at the end of the week, I *had* been paying $10/mb to
send traffic through transit to reach the whole rest of the
internet.  Now, I'm paying $5+$4+$4+$5+$2, or $30, and
I don't have a full set of routes, so I've still got to keep
paying the transit provider as well at $10.

If this is the math you are using to justify your stance it's probably
worth reconsidering. You ignore that each of those if sent through transit
would have been $10 so the cost of $5, $4, and $2/Mb represent a savings of
$5, $6,and $8. Why would you add them? Sure there are factors you have to
evaluate like putting yourself under a minimum commit with $transit or if
the amount of traffic is worth peering over but you would generally have to
make those evaluations for peering anyway. The real difference is the
volume of traffic needed before a $2/mb savings is worth peering directly
for is higher than if the savings were the full $10 but that doesn't mean
its never worth it. There is a difference between saying I did the math
and transit remains the cheaper option and saying Paid peering would save
us both money and improve performance at the same time but we refuse to do
it anyway on principal.

The concept of fair gets brought up a lot when talking about the ability of
a startup to come in to compete against bigger players in the content space
but really what do you think the impact is if the largest established
content providers peer freely and smaller newcomers only have paid options
available for traffic?



Some other things I also want to get to:
On Vi's analogy vs Amazon prime

One major different I think people overlook is overusing Amazon prime would
mean buying too many things from Amazon. Even when you purchase through
companies selling through Amazon they get a cut of the sale and some of
that I assume gets applied to covering any additional shipping costs not
covered by Prime. If Internet traffic used the same model would ISP's
receive a portion of proceeds for ad revenue on places like Youtube or a
percentage of Netflix subscription fees? I'm not making the case that thats
the model that should be used I'm only pointing out that analogies are best
to break things down into simple terms for people but have diminishing
returns in usefulness when getting into details.

The other problem with Vi's analogy is the shipping company delivers to the
driveway of the customer where a more real life scenario would be something
closer to Amazon having a distribution center in that city, and both
Comcast and FedEx are already both sitting idle in the parking lot. Amazon
pays FedEx to give the package to Comcast in the next parking space, who
then drives it to the customers house. Comcast says to Netflix, since I am
the one driving this from the parking lot to the customers house, why not
just pay me instead of paying FedEx more money to just put it on my truck?
Amazon says, but FedEx will deliver the package to France if I tell them
to. Comcast says, but you don't even serve france out of this distribution
center, and I am not asking to be charged for all packages, only the ones I
deliver instead of FedEx. Amazon says, you are right, we have technology to
give your packages directly to you and stuff going to France to Fedex, and
it would be best for both of us to do it, but unless you'll deliver my
packages for free I'm going to keep paying FedEx to just keep loading them
on your truck. Comcast says have at it, there are 5 trucks for FedEx to
load freely now but if you need more you have to compromise with us on a
deal that works better for both of us. Amazon says, when we are done with
you in the media we won't need to compromise.


Government regulation of interconnects

I agree with

Re: Observations of an Internet Middleman (Level3)

2014-05-16 Thread Livingood, Jason

On 5/15/14, 12:49 PM, arvindersi...@mail2tor.com
arvindersi...@mail2tor.com wrote:


I have two issues with the comments:

2. You mention that all packets treated equally - no games.  Why does
AS7922 assign the speed test different DSCP from regular internet
connection?

I have no idea what you are talking about. Our Internet traffic, including
to speedtest web sites, is all best effort class data. Do you have more
specific information?

Jason

Re: Observations of an Internet Middleman (Level3)

2014-05-16 Thread Vinny Abello


On , Livingood, Jason wrote:

On 5/15/14, 12:49 PM, arvindersi...@mail2tor.com
arvindersi...@mail2tor.com wrote:



I have two issues with the comments:

2. You mention that all packets treated equally - no games.  Why does
AS7922 assign the speed test different DSCP from regular internet
connection?


I have no idea what you are talking about. Our Internet traffic, 
including

to speedtest web sites, is all best effort class data. Do you have more
specific information?

Jason


I think he's questioning why packets from speedtest.comcast.net have CS1 
if everything is supposedly equal, and what that is used for. A quick 
Wireshark shows that to be true right now running to your Plainfield, NJ 
speedtest site, and my network peers directly with Comcast.


I'm kind of curious too. What is the purpose of this? Is it the 
traditional purpose of CS1 to be less than best effort or something 
else? If this is the case it seems Comcast would be purposely putting 
themselves at a disadvantage in speed tests when congestion is 
involved... or is this possibly on purpose to make peering problems look 
even worse during congestion?


-Vinny

Re: Observations of an Internet Middleman (Level3) (was: RIP

2014-05-16 Thread Owen DeLong

All the talk about ratios is a red herring… The real issue boils down to this:

1.  The access (eyeball) networks don’t want to bear the cost of delivering 
what they promised to their customers.
2.  This is because when they built their business models, they didn’t 
expect their customers to use nearly as much
of their promised bandwidth as they are now using. Most of the models 
were constructed around the idea that
a customer receiving, say 27mbps down/7mbps up would use all of that 
bandwidth in short bursts and mostly
use less than a megabit.
3.  New services have been developed (streaming video, et al.) which have 
created an increasing demand from
customers for more of the bandwidth they were sold.
4.  Instead of raising the prices to the access network customers or 
accepting that the lavish profits that they eyeball
networks had been pocketing were no more, the access networks are 
trying to slough off the costs of delivering
that higher fraction of what they sold onto someone else.
5.  The content providers looked like an easy target with the advantage 
that:
A.  Some of them appear to have deep pockets.
B.  They are the competition for many of the access network’s other 
lines of business, so increasing their costs
helps make them less competitive.
C.  Consumers are emotional about price increases. Content 
providers look at it as a business problem and
perform a mathematical analysis. If their customer satisfaction 
impact costs more than paying the extortion
from the access networks, they’ll pay it.

In reality, if the $ACCESS_PROVIDERS wanted to satisfy their customers, they’d 
be aggressively seeking to peer with content providers in as many locations as 
possible. They might (reasonably) require content providers to build out to 
additional locations to keep their long-haul costs down (It’s reasonable, IMHO, 
for a content provider not to want to carry multiple gigabits of traffic from a 
content provider clear across the country for free. If $CONTENT_PROVIDER wants 
to access California customers of $ACCESS_PROVIDER, then it’s reasonable for 
$ACCESS_PROVIDER to insist that $CONTENT_PROVIDER peer in California for 
delivering those bits.)

Neither side of this issue has completely clean hands. Both have been trying to 
take as much of the money on the table for themselves with limited regard for 
serving the consumer. The Access Networks have done a far worse job of serving 
the consumer than the content providers and that’s a big part of what is 
driving the current backlash. As a general rule, access customers don’t select 
the provider they love the most, they select the one they think sucks the least.

I think the recent FCC NPRM is a bit optimistic in that it expects the 
$ACCESS_PROVIDERS to act in good faith. If they do, it will likely turn out to 
be a limited victory for the $ACCESS_PROVIDERS. However, I don’t expect the 
$ACCESS_PROVIDERS to live within that limited victory. Assuming the NRPM 
becomes rule and then withstands the likely legal challenges, I expect they 
will, as usual, play in the gray areas of the ruling as much as they think they 
legally can and push the edges as far as possible to try and extort every 
dollar they can from $CONTENT_PROVIDERS with this so-called fast-lane (which we 
all know is just preferential peering and/or QoS[1] tuning).  I suspect they 
will likely push this far enough that over the next several years, things will 
get progressively worse until the FCC finally decides that they have to move 
from section 706 to Title II.

OTOH, if I’m wrong and the $ACCESS_PROVIDERS suddenly start behaving like 
civilized companies, develop a sudden concern for their customers’ experiences, 
and start unimaginably acting in good faith, the proposed rule wouldn’t be so 
bad for $CONTENT_PROVIDERS, $CONSUMERS, or $ACCESS_PROVIDERS. 

Of course, you can already see the $ACCESS_PROVIDERS laying the groundwork to 
try and mount a legal challenge against the FCC’s authority to use rule 706. 
Sadly, some of this groundwork is being laid by FCC commissioners. Said 
commissioners clearly have no interest in representing the people’s interest 
and are strictly there as mouth-pieces for some of the big players in the 
industry.

Owen

[1] QoS — A deceptive name if ever there was one. QoS is not about Quality of 
Service, it’s about screwing over network users by choice rather than by chance 
when you haven’t built an adequate network.

Re: Observations of an Internet Middleman (Level3)

On Fri, May 16, 2014 at 7:56 AM, Vinny Abello vi...@abellohome.net wrote:

 I think he's questioning why packets from speedtest.comcast.net have CS1 if
 everything is supposedly equal, and what that is used for. A quick Wireshark
 shows that to be true right now running to your Plainfield, NJ speedtest
 site, and my network peers directly with Comcast.

are you measuring inside the (for this) comcast network or after your
cable-modem?
I recall that the cable-modem(s) often (by docsis config) impose some
qos markings on the lan-side of the connection.

I think they can do the same on the WAN side for traffic leaving your
site to the tubes... but you probably can't measure that as easily as
with wireshark on your pc.

-chris
(also, is there some other equipment between your wiresharker and the
cable-modem? could that equipment be re-marking/marking as well?)

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality (was: Wow its been quiet here...

2014-05-16 Thread Owen DeLong


On May 16, 2014, at 3:25 AM, Rick Astley jna...@gmail.com wrote:

 Broadband is too expensive in the US compared to other places
 
 I have seen this repeated so many times that I assume it's true but I have
 never seen anything objective as to why. I can tell you if you look at
 population density by country the US is 182nd in the world and the average
 broadband speed (based on OOKLA:
 http://www.netindex.com/download/allcountries/) is 30th in the world. South
 Korea that is well known for its fast broadband speeds has a density of
 505/km vs the US at 32/km. We have about 1/15 of the population density and
 about 1/2 the average broadband speed. Hong Kong, Singapore, Netherlands,
 Japan, Macau etc. all have more than 10x the population density in the US
 so definitely not all countries with fast broadband make for a fair
 comparison and there are likely fewer that do. The UK is only beating the
 US by 2Mbps but has a population density of 262/km.
 
 So while its a fair assessment that broadband in the US is very bias to
 ignore some of the other factors involved. Another mistake I see people
 keep making is in comparing the cost of broadband in the US in $USD to
 other countries around the world. The cost of broadband in Estonia is only
 about $30/month. OMG, I can't believe broadband is cheaper in Estonia! What
 people ignore is everything is cheaper in Estonia, the average household
 income in Estonia is $14k vs $55k here. By that measure broadband is more
 expensive for families there than it is in the US. This is another point
 people repeat without bothering to qualify. This would be like my
 grandfather comparing the costs of a candy bar from back when he was a kid
 to today but ignoring inflation.

I might be willing to accept this argument if it weren’t for the fact that rural
locations in the US are far more likely to have FTTH than higher density areas
because the whole USF thing has inverted the priorities.

I live in the largest city in the bay area, yet there is only one facilities
based provider in my area that can deliver 2mbps or more and that’s over
HFC. Twisted pair is abysmal and there is no fiber.

The situation is not significantly better in the densest city in the bay area, 
either.

South Korea averages 4x US Speed for an average $28.50/month.
US averages 1x US Speed for an average $45.50/month.
(http://edition.cnn.com/2010/TECH/03/31/broadband.south.korea/)

Korean average annual wage: $36,757 @ 21% tax = $29,038 take-home.
US Average annual wage: $55,048 @ 29.6% tax = $38,753 take-home.
(http://en.wikipedia.org/wiki/List_of_countries_by_average_wage)

So that says KR take-home wage = ~75% of US wage.
75% of $45.50 is $34.125

So 4x speed is still approximately $5 cheaper per month in KR than in the US.

Owen

Re: Observations of an Internet Middleman (Level3)

2014-05-16 Thread Livingood, Jason

On 5/16/14, 7:56 AM, Vinny Abello vi...@abellohome.net wrote:

I think he's questioning why packets from speedtest.comcast.net have CS1
if everything is supposedly equal, and what that is used for. A quick
Wireshark shows that to be true right now running to your Plainfield, NJ
speedtest site, and my network peers directly with Comcast.

I'm kind of curious too. What is the purpose of this? Is it the
traditional purpose of CS1 to be less than best effort or something
else? If this is the case it seems Comcast would be purposely putting
themselves at a disadvantage in speed tests when congestion is
involved... or is this possibly on purpose to make peering problems look
even worse during congestion?

Ah! That makes sense now. CS1 is used internally to mark best effort
Internet traffic. This has often caused confusion when folks see our
markings. If folks want to send me any data off-list that you think merits
further investigation, let me know (never know if something someplace is
an honest config error).

Jason

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Friday, May 16, 2014 03:54:33 PM Owen DeLong wrote:

 customers. 2. This is because when they built their
 business models, they didn’t expect their customers to
 use nearly as much of their promised bandwidth as they
 are now using. Most of the models were constructed
 around the idea that a customer receiving, say 27mbps
 down/7mbps up would use all of that bandwidth in short
 bursts and mostly use less than a megabit.

And in general, models have assumed, for a long time, that 
customer demand patterns are largely asymmetric.

While that is true a lot of the time (especially for eyeball 
networks), it is less so now due to social media. Social 
media forces the use of symmetric bandwidth (like FTTH), 
putting even more demand on the network, and making the gist 
of this thread an even bigger issue, if you discount the 
fact, of course, that Broadband in the U.S. currently sucks 
for a developed market.

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Observations of an Internet Middleman (Level3) (was: RIP

Social media is not a big driver of symmetrical traffic here in the US or
internationally.  Broadband suffers here for a number of reasons, mainly
topological and population density, in comparison to places like Japan,
parts (but certainly not all) of Europe, and South Korea.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 11:02 AM, Mark Tinka mark.ti...@seacom.mu wrote:

 On Friday, May 16, 2014 03:54:33 PM Owen DeLong wrote:

  customers. 2. This is because when they built their
  business models, they didn’t expect their customers to
  use nearly as much of their promised bandwidth as they
  are now using. Most of the models were constructed
  around the idea that a customer receiving, say 27mbps
  down/7mbps up would use all of that bandwidth in short
  bursts and mostly use less than a megabit.

 And in general, models have assumed, for a long time, that
 customer demand patterns are largely asymmetric.

 While that is true a lot of the time (especially for eyeball
 networks), it is less so now due to social media. Social
 media forces the use of symmetric bandwidth (like FTTH),
 putting even more demand on the network, and making the gist
 of this thread an even bigger issue, if you discount the
 fact, of course, that Broadband in the U.S. currently sucks
 for a developed market.

 Mark.

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Friday, May 16, 2014 05:08:33 PM Scott Helms wrote:

 Social media is not a big driver of symmetrical traffic
 here in the US or internationally.  Broadband suffers
 here for a number of reasons, mainly topological and
 population density, in comparison to places like Japan,
 parts (but certainly not all) of Europe, and South
 Korea.

It might not be (now), but if symmetrical bandwidth will go 
in on the back of teenagers wanting to upload videos about 
their lives, the meer fact that the bandwidth is there means 
someone will find bigger and better use for it, than social 
media.

We saw this when we deployed FTTH in Malaysia, back in '09.

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Observations of an Internet Middleman (Level3) (was: RIP

2014-05-16 Thread Jay Ashworth

- Original Message -
 From: Mark Tinka mark.ti...@seacom.mu

 While that is true a lot of the time (especially for eyeball
 networks), it is less so now due to social media. Social
 media forces the use of symmetric bandwidth (like FTTH),
 putting even more demand on the network, 

Oh yes; clearly, Twitter will be the end of L3.

:-)

Could you expand a bit, Mark on Social media forces the use of symmetric
bandwidth?  Which social media platform is it that you think has a)
symmetrical flows that b) are big enough to figure into transit symmetry?

Cheers,
-- jra
-- 
Jay R. Ashworth  Baylink   j...@baylink.com
Designer The Things I Think   RFC 2100
Ashworth  Associates   http://www.bcp38.info  2000 Land Rover DII
St Petersburg FL USA  BCP38: Ask For It By Name!   +1 727 647 1274

Re: Observations of an Internet Middleman (Level3) (was: RIP

Mark,

Bandwidth use trends are actually increasingly asymmetical because of the
popularity of OTT video.

Social media, even with video uploading, simply doesn't generate that much
traffic per session.

During peak period, Real-Time Entertainment traffic is by far the most
dominant traffic category, accounting for almost
half of the downstream bytes on the network. As observed in past reports,
Social Networking applications continue to
be very well represented on the mobile network. This speaks to their
popularity with subscribers as these applications
typically generate far less traffic than those that stream audio and video.

https://www.sandvine.com/downloads/general/global-internet-phenomena/2013/sandvine-global-internet-phenomena-report-1h-2013.pdf

Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms

On Fri, May 16, 2014 at 11:26 AM, Mark Tinka mark.ti...@seacom.mu wrote:

On Friday, May 16, 2014 05:08:33 PM Scott Helms wrote:

Social media is not a big driver of symmetrical traffic
here in the US or internationally. Broadband suffers
here for a number of reasons, mainly topological and
population density, in comparison to places like Japan,
parts (but certainly not all) of Europe, and South
Korea.

It might not be (now), but if symmetrical bandwidth will go
in on the back of teenagers wanting to upload videos about
their lives, the meer fact that the bandwidth is there means
someone will find bigger and better use for it, than social
media.

We saw this when we deployed FTTH in Malaysia, back in '09.

Mark.

Re: Observations of an Internet Middleman (Level3) (was: RIP



Jay Ashworth wrote the following on 5/16/2014 10:35 AM:

- Original Message -

From: Mark Tinka mark.ti...@seacom.mu
While that is true a lot of the time (especially for eyeball
networks), it is less so now due to social media. Social
media forces the use of symmetric bandwidth (like FTTH),
putting even more demand on the network,

Oh yes; clearly, Twitter will be the end of L3.

:-)

Could you expand a bit, Mark on Social media forces the use of symmetric
bandwidth?  Which social media platform is it that you think has a)
symmetrical flows that b) are big enough to figure into transit symmetry?

Cheers,
-- jra
Applications like Skype and Facetime (especially conference calls) would 
be one example where an application benefits from symmetric (or 
asymmetric in favor of higher upload speed) connectivity. Cloud office 
applications like storage of documents, email, and IVR telephony also 
benefit from symmetrical connectivity. Off-site backup software is 
another great example. Most residential connections are ill suited for 
this. I believe these applications (and derivatives) would be more 
popular today if the connectivity was available.


--Blake

Re: Observations of an Internet Middleman (Level3) (was: RIP

Blake,

None of those applications come close to causing symmetrical traffic
patterns and for many/most networks the upstream connectivity has greatly
improved.  Anything related to voice is no more than 80 kbps per line, even
if the SIP traffic isn't trunked (less if it is because the signaling data
is shared).  Document sharing is not being impinged, on my residential
account right now I've uploaded about 30 documents this morning including
large PDFs and Power Point presentations.

Off site back up is one use that could drive traffic, but I don't believe
that the limiting factor is bandwidth.  We looked at getting into that
business and from what we saw the limiting factor was that most residential
and SOHO accounts didn't want to pay enough to cover your storage 
management costs.  In our analysis the impact of bandwidth on the consumer
side adoption was basically zero.  There is no expectation that back ups
run instantly.  Having said all of that, even if hosted back up became
wildly popular would not change the balance of power because OTT video is
both larger, especially for HD streams, and used much more frequently.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 11:53 AM, Blake Hudson bl...@ispn.net wrote:


 Jay Ashworth wrote the following on 5/16/2014 10:35 AM:

 - Original Message -

 From: Mark Tinka mark.ti...@seacom.mu
 While that is true a lot of the time (especially for eyeball
 networks), it is less so now due to social media. Social
 media forces the use of symmetric bandwidth (like FTTH),
 putting even more demand on the network,

 Oh yes; clearly, Twitter will be the end of L3.

 :-)

 Could you expand a bit, Mark on Social media forces the use of symmetric
 bandwidth?  Which social media platform is it that you think has a)
 symmetrical flows that b) are big enough to figure into transit symmetry?

 Cheers,
 -- jra

 Applications like Skype and Facetime (especially conference calls) would
 be one example where an application benefits from symmetric (or asymmetric
 in favor of higher upload speed) connectivity. Cloud office applications
 like storage of documents, email, and IVR telephony also benefit from
 symmetrical connectivity. Off-site backup software is another great
 example. Most residential connections are ill suited for this. I believe
 these applications (and derivatives) would be more popular today if the
 connectivity was available.

 --Blake

Re: Observations of an Internet Middleman (Level3) (was: RIP

Certainly video is one of the most bandwidth intensive applications. I 
don't deny that a  1 Mbps video call is both less common and consumes 
less bandwidth than an 8Mbps HD stream. However, if Americans had access 
to symmetric connections capable of reliably making HD video calls (they 
don't, in my experience), we might be seeing video calls as a common 
occurrence and not a novelty. I think the state of usage is a reflection 
on the technology available.


If the capability was available at an affordable price to residential 
consumers, we might see those consumers stream movies or send videos 
from their home or mobile devices via their internet connection directly 
to the recipient rather than through a centralized source like Disney, 
NetFlix, Youtube, etc. Video sharing sites (like youtube, vimeo, etc) 
primary reason for existence is due to the inability of the site's users 
to distribute content themselves. One of the hurdles to overcome in 
video sharing is the lack of availability in affordable internet 
connectivity that is capable of sending video at reasonable (greater 
than real time) speeds.


--Blake

Scott Helms wrote the following on 5/16/2014 11:02 AM:

Blake,

None of those applications come close to causing symmetrical traffic 
patterns and for many/most networks the upstream connectivity has 
greatly improved.  Anything related to voice is no more than 80 kbps 
per line, even if the SIP traffic isn't trunked (less if it is because 
the signaling data is shared).  Document sharing is not being 
impinged, on my residential account right now I've uploaded about 30 
documents this morning including large PDFs and Power Point presentations.


Off site back up is one use that could drive traffic, but I don't 
believe that the limiting factor is bandwidth.  We looked at getting 
into that business and from what we saw the limiting factor was that 
most residential and SOHO accounts didn't want to pay enough to cover 
your storage  management costs.  In our analysis the impact of 
bandwidth on the consumer side adoption was basically zero.  There is 
no expectation that back ups run instantly.  Having said all of that, 
even if hosted back up became wildly popular would not change the 
balance of power because OTT video is both larger, especially for HD 
streams, and used much more frequently.



Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 11:53 AM, Blake Hudson bl...@ispn.net 
mailto:bl...@ispn.net wrote:



Jay Ashworth wrote the following on 5/16/2014 10:35 AM:

- Original Message -

From: Mark Tinka mark.ti...@seacom.mu
mailto:mark.ti...@seacom.mu
While that is true a lot of the time (especially for eyeball
networks), it is less so now due to social media. Social
media forces the use of symmetric bandwidth (like FTTH),
putting even more demand on the network,

Oh yes; clearly, Twitter will be the end of L3.

:-)

Could you expand a bit, Mark on Social media forces the use
of symmetric
bandwidth?  Which social media platform is it that you think
has a)
symmetrical flows that b) are big enough to figure into
transit symmetry?

Cheers,
-- jra

Applications like Skype and Facetime (especially conference calls)
would be one example where an application benefits from symmetric
(or asymmetric in favor of higher upload speed) connectivity.
Cloud office applications like storage of documents, email, and
IVR telephony also benefit from symmetrical connectivity. Off-site
backup software is another great example. Most residential
connections are ill suited for this. I believe these applications
(and derivatives) would be more popular today if the connectivity
was available.

--Blake

Re: Observations of an Internet Middleman (Level3) (was: RIP

Blake,

I might agree with your premise if weren't for a couple of items.

1)  Very few consumers are walking around with a HD or 4K camera today.

2)  Most consumers who want to share video wouldn't know how to host it
themselves, which isn't an insurmountable issue but is a big barrier to
entry especially given the number of NAT'ed connections.  I think this is
much more of a problem than available bandwidth.

3)  Most consumers who want to share videos seem to be satisfied with
sharing via one of the cloud services whether that be YouTube (which was
created originally for that use), Vimeo, or one of the other legions of
services like DropBox.

4)  Finally, upstream bandwidth has increased on many/most operators.  I
just ran the FCC's speedtest (mLab not Ookla) and got 22 mbps on my
residential cable internet service.  I subscribe to one of the major MSOs
for a normal residential package.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 12:38 PM, Blake Hudson bl...@ispn.net wrote:

 Certainly video is one of the most bandwidth intensive applications. I
 don't deny that a  1 Mbps video call is both less common and consumes less
 bandwidth than an 8Mbps HD stream. However, if Americans had access to
 symmetric connections capable of reliably making HD video calls (they
 don't, in my experience), we might be seeing video calls as a common
 occurrence and not a novelty. I think the state of usage is a reflection on
 the technology available.

 If the capability was available at an affordable price to residential
 consumers, we might see those consumers stream movies or send videos from
 their home or mobile devices via their internet connection directly to the
 recipient rather than through a centralized source like Disney, NetFlix,
 Youtube, etc. Video sharing sites (like youtube, vimeo, etc) primary reason
 for existence is due to the inability of the site's users to distribute
 content themselves. One of the hurdles to overcome in video sharing is the
 lack of availability in affordable internet connectivity that is capable of
 sending video at reasonable (greater than real time) speeds.

 --Blake

 Scott Helms wrote the following on 5/16/2014 11:02 AM:

 Blake,

 None of those applications come close to causing symmetrical traffic
 patterns and for many/most networks the upstream connectivity has greatly
 improved.  Anything related to voice is no more than 80 kbps per line, even
 if the SIP traffic isn't trunked (less if it is because the signaling data
 is shared).  Document sharing is not being impinged, on my residential
 account right now I've uploaded about 30 documents this morning including
 large PDFs and Power Point presentations.

 Off site back up is one use that could drive traffic, but I don't believe
 that the limiting factor is bandwidth.  We looked at getting into that
 business and from what we saw the limiting factor was that most residential
 and SOHO accounts didn't want to pay enough to cover your storage 
 management costs.  In our analysis the impact of bandwidth on the consumer
 side adoption was basically zero.  There is no expectation that back ups
 run instantly.  Having said all of that, even if hosted back up became
 wildly popular would not change the balance of power because OTT video is
 both larger, especially for HD streams, and used much more frequently.


 Scott Helms
 Vice President of Technology
 ZCorum
 (678) 507-5000
 
 http://twitter.com/kscotthelms
 


 On Fri, May 16, 2014 at 11:53 AM, Blake Hudson bl...@ispn.net mailto:
 bl...@ispn.net wrote:


 Jay Ashworth wrote the following on 5/16/2014 10:35 AM:

 - Original Message -

 From: Mark Tinka mark.ti...@seacom.mu
 mailto:mark.ti...@seacom.mu
 While that is true a lot of the time (especially for eyeball
 networks), it is less so now due to social media. Social
 media forces the use of symmetric bandwidth (like FTTH),
 putting even more demand on the network,

 Oh yes; clearly, Twitter will be the end of L3.

 :-)

 Could you expand a bit, Mark on Social media forces the use
 of symmetric
 bandwidth?  Which social media platform is it that you think
 has a)
 symmetrical flows that b) are big enough to figure into
 transit symmetry?

 Cheers,
 -- jra

 Applications like Skype and Facetime (especially conference calls)
 would be one example where an application benefits from symmetric
 (or asymmetric in favor of higher upload speed) connectivity.
 Cloud office applications like storage of documents, email, and
 IVR telephony also benefit from symmetrical connectivity. Off-site
 backup software is another great example.

Re: Observations of an Internet Middleman (Level3)


Scott Helms wrote:

Mark,

Bandwidth use trends are actually increasingly asymmetical because of the
popularity of OTT video.


Until my other half decides to upload a video.

Is it too much to ask for a bucket of bits that I can use in whichever 
direction happens
to be needed at the moment?

Mike

Re: Observations of an Internet Middleman (Level3) (was: RIP

Thanks for the insight Scott. I appreciate the experience and point of 
view you're adding to this discussion (not just the responses to me). 
While I might be playing the devil's advocate here a bit, I think one 
could argue each of the points you've made below.


I do feel that general usage patterns are a reflection of the 
technologies that have traditionally been available to consumers. New 
uses and applications would be available to overcome hurdles if the 
technologies had developed to be symmetrical. I'm not saying that the 
asymmetrical choice was a bad one, but it was not without consequences. 
If residential ISPs sell asymmetric connections for decades, how can the 
ISP expect that application developers would not take this into account 
when developing applications? I don't think my application would be very 
successful if it required X Mbps and half of my market did not meet this 
requirement. Of course content/service providers are going to tailor 
their services based around their market.


--Blake

Scott Helms wrote the following on 5/16/2014 12:06 PM:

Blake,

I might agree with your premise if weren't for a couple of items.

1)  Very few consumers are walking around with a HD or 4K camera today.

2)  Most consumers who want to share video wouldn't know how to host 
it themselves, which isn't an insurmountable issue but is a big 
barrier to entry especially given the number of NAT'ed connections.  I 
think this is much more of a problem than available bandwidth.


3)  Most consumers who want to share videos seem to be satisfied with 
sharing via one of the cloud services whether that be YouTube (which 
was created originally for that use), Vimeo, or one of the other 
legions of services like DropBox.


4)  Finally, upstream bandwidth has increased on many/most operators. 
 I just ran the FCC's speedtest (mLab not Ookla) and got 22 mbps on my 
residential cable internet service.  I subscribe to one of the major 
MSOs for a normal residential package.



Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 12:38 PM, Blake Hudson bl...@ispn.net 
mailto:bl...@ispn.net wrote:


Certainly video is one of the most bandwidth intensive
applications. I don't deny that a  1 Mbps video call is both less
common and consumes less bandwidth than an 8Mbps HD stream.
However, if Americans had access to symmetric connections capable
of reliably making HD video calls (they don't, in my experience),
we might be seeing video calls as a common occurrence and not a
novelty. I think the state of usage is a reflection on the
technology available.

If the capability was available at an affordable price to
residential consumers, we might see those consumers stream movies
or send videos from their home or mobile devices via their
internet connection directly to the recipient rather than through
a centralized source like Disney, NetFlix, Youtube, etc. Video
sharing sites (like youtube, vimeo, etc) primary reason for
existence is due to the inability of the site's users to
distribute content themselves. One of the hurdles to overcome in
video sharing is the lack of availability in affordable internet
connectivity that is capable of sending video at reasonable
(greater than real time) speeds.

--Blake

Scott Helms wrote the following on 5/16/2014 11:02 AM:

Blake,

None of those applications come close to causing symmetrical
traffic patterns and for many/most networks the upstream
connectivity has greatly improved.  Anything related to voice
is no more than 80 kbps per line, even if the SIP traffic
isn't trunked (less if it is because the signaling data is
shared).  Document sharing is not being impinged, on my
residential account right now I've uploaded about 30 documents
this morning including large PDFs and Power Point presentations.

Off site back up is one use that could drive traffic, but I
don't believe that the limiting factor is bandwidth.  We
looked at getting into that business and from what we saw the
limiting factor was that most residential and SOHO accounts
didn't want to pay enough to cover your storage  management
costs.  In our analysis the impact of bandwidth on the
consumer side adoption was basically zero.  There is no
expectation that back ups run instantly.  Having said all of
that, even if hosted back up became wildly popular would not
change the balance of power because OTT video is both larger,
especially for HD streams, and used much more frequently.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000 tel:%28678%29%20507-5000

Re: Observations of an Internet Middleman (Level3)

Michael,

No, its not too much to ask and any end user who has that kind of
requirement can order a business service to get symmetrical service but the
reality is that symmetrical service costs more and the vast majority of
customers don't use the upstream capacity they have today.  I have personal
insight into about half a million devices and the percentage of people who
bump up against their upstream rate is less than 0.2%.  I have the ability
to get data on another 10 million and the last time I checked their rates
were similar.

This kind of question has been asked of operators since long before cable
companies could offer internet service.  What happens if everyone in an
area use their telephone (cellular or land line) at the same time?  A fast
busy or recorded All circuits are busy message.  Over subscription is a
fact of economics in virtually everything we do.  By this logic restaurants
should be massively over built so that there is never a waiting line,
highways should always be a speed limit ride, and all of these things would
cost much more money than they do today.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Sun, Apr 27, 2014 at 8:21 PM, Michael Thomas m...@mtcc.com wrote:

 Scott Helms wrote:

 Mark,

 Bandwidth use trends are actually increasingly asymmetical because of the
 popularity of OTT video.


 Until my other half decides to upload a video.

 Is it too much to ask for a bucket of bits that I can use in whichever
 direction happens
 to be needed at the moment?

 Mike

Re: Observations of an Internet Middleman (Level3)

2014-05-16 Thread Matthew Petach

On Sun, Apr 27, 2014 at 5:21 PM, Michael Thomas m...@mtcc.com wrote:

 Scott Helms wrote:

 Mark,

 Bandwidth use trends are actually increasingly asymmetical because of the
 popularity of OTT video.


 Until my other half decides to upload a video.

 Is it too much to ask for a bucket of bits that I can use in whichever
 direction happens
 to be needed at the moment?

 Mike


Sure, I've got two of those; they're called T1 lines,
and they work equally well in both directions, even
when the other half wants to upload cat videos.

Matt

Re: Observations of an Internet Middleman (Level3) (was: RIP

Blake,

You're absolutely correct.  The world adapts to the reality that we find
ourselves in via normal market mechanics.  The problem with proposing that
connectivity for residential customers should be more symmetrical is that
its expensive, which is why we as operators didn't roll it out that way to
start.  We also don't see consumer demand for symmetrical connections and
with the decline in peer to peer file sharing we've actually seen a
decrease the ratio of used upstream bandwidth (though not a decrease in
absolute terms).

I would like to deliver symmetrical bandwidth to all consumers just so
those few customers who need it today would have lower bills but trying to
justify that to our CFO without being able to point to an increase in
revenue either because of more revenue per sub or more subs is a very tough
task.  I don't believe my situation is uncommon.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 1:20 PM, Blake Hudson bl...@ispn.net wrote:

 Thanks for the insight Scott. I appreciate the experience and point of
 view you're adding to this discussion (not just the responses to me). While
 I might be playing the devil's advocate here a bit, I think one could argue
 each of the points you've made below.

 I do feel that general usage patterns are a reflection of the technologies
 that have traditionally been available to consumers. New uses and
 applications would be available to overcome hurdles if the technologies had
 developed to be symmetrical. I'm not saying that the asymmetrical choice
 was a bad one, but it was not without consequences. If residential ISPs
 sell asymmetric connections for decades, how can the ISP expect that
 application developers would not take this into account when developing
 applications? I don't think my application would be very successful if it
 required X Mbps and half of my market did not meet this requirement. Of
 course content/service providers are going to tailor their services based
 around their market.

 --Blake

 Scott Helms wrote the following on 5/16/2014 12:06 PM:

 Blake,

 I might agree with your premise if weren't for a couple of items.

 1)  Very few consumers are walking around with a HD or 4K camera today.

 2)  Most consumers who want to share video wouldn't know how to host it
 themselves, which isn't an insurmountable issue but is a big barrier to
 entry especially given the number of NAT'ed connections.  I think this is
 much more of a problem than available bandwidth.

 3)  Most consumers who want to share videos seem to be satisfied with
 sharing via one of the cloud services whether that be YouTube (which was
 created originally for that use), Vimeo, or one of the other legions of
 services like DropBox.

 4)  Finally, upstream bandwidth has increased on many/most operators.  I
 just ran the FCC's speedtest (mLab not Ookla) and got 22 mbps on my
 residential cable internet service.  I subscribe to one of the major MSOs
 for a normal residential package.


 Scott Helms
 Vice President of Technology
 ZCorum
 (678) 507-5000
 
 http://twitter.com/kscotthelms
 


 On Fri, May 16, 2014 at 12:38 PM, Blake Hudson bl...@ispn.net mailto:
 bl...@ispn.net wrote:

 Certainly video is one of the most bandwidth intensive
 applications. I don't deny that a  1 Mbps video call is both less
 common and consumes less bandwidth than an 8Mbps HD stream.
 However, if Americans had access to symmetric connections capable
 of reliably making HD video calls (they don't, in my experience),
 we might be seeing video calls as a common occurrence and not a
 novelty. I think the state of usage is a reflection on the
 technology available.

 If the capability was available at an affordable price to
 residential consumers, we might see those consumers stream movies
 or send videos from their home or mobile devices via their
 internet connection directly to the recipient rather than through
 a centralized source like Disney, NetFlix, Youtube, etc. Video
 sharing sites (like youtube, vimeo, etc) primary reason for
 existence is due to the inability of the site's users to
 distribute content themselves. One of the hurdles to overcome in
 video sharing is the lack of availability in affordable internet
 connectivity that is capable of sending video at reasonable
 (greater than real time) speeds.

 --Blake

 Scott Helms wrote the following on 5/16/2014 11:02 AM:

 Blake,

 None of those applications come close to causing symmetrical
 traffic patterns and for many/most networks the upstream
 connectivity has greatly improved.  Anything related to voice
 is no more than 80 kbps per line, even if the SIP traffic
 isn't trunked (less if it is

Re: Observations of an Internet Middleman (Level3)

2014-05-16 Thread Laszlo Hanyecz

I'd just like to point out that a lot of people are in fact using their 
upstream capability, and the operators always throw a fit and try to cut off 
specific applications to force it back into the idle state.  For example P2P 
things like torrents and most recently the open NTP and DNS servers.  How about 
SMTP?  Not sure about you guys but my local broadband ISP has cut me off and 
told me that my 'unlimited internet' is in fact limited.  The reality is that 
those people who are not using it (99.8%?) are just being ripped off - paying 
for something they were told they need, thinking that it's there when they want 
it, then getting cut off when they actually try to use it.

It's not like whining about it here will change anything, but the prices are 
severely distorted.  Triple play packages are designed to force people to pay 
for stuff they don't need or want - distorting the price of a service hoping to 
recover it elsewhere, then if the gamble doesn't pan out, the customer loses 
again.  The whole model is based on people buying stuff that they won't 
actually come to collect, so then you can sell it an infinite number of times.  
The people who do try to collect what was sold to them literally end up getting 
called names and cut off - terms like excessive bandwidth user and network 
abuser are used to describe paying customers.  With regard to the peering 
disputes, it's hardly surprising that their business partners are treated with 
the same attitude as their customers.  Besides, if you cut off the customers 
and peers who are causing that saturation, then the existing peering links can 
support an infinite number of idle subscribers.  The next phase is 
usage-based-billing which is kind of like having to pay a fine for using it, so 
they can artificially push the price point lower and hopefully get some more 
idle customers.  That will help get the demand down and keep the infrastructure 
nice and idle.  When you're paying for every cat video maybe you realize you 
can live without it instead.

Everyone has been trained so well, they don't even flinch anymore when they 
hear about over subscription, and they apologize for the people who are doing 
it to them.  The restaurant analogy is incorrect - you can go to the restaurant 
next door if a place is busy, thus they have pressure to increase their 
capacity if they want to sell more meals.  With broadband you can't go anywhere 
else, (for most people) there's only one restaurant, and there's a week long 
waiting list.  If you don't like it, you're probably an abuser or excessive 
eater anyway.

-Laszlo


On May 16, 2014, at 5:34 PM, Scott Helms khe...@zcorum.com wrote:

 Michael,
 
 No, its not too much to ask and any end user who has that kind of
 requirement can order a business service to get symmetrical service but the
 reality is that symmetrical service costs more and the vast majority of
 customers don't use the upstream capacity they have today.  I have personal
 insight into about half a million devices and the percentage of people who
 bump up against their upstream rate is less than 0.2%.  I have the ability
 to get data on another 10 million and the last time I checked their rates
 were similar.
 
 This kind of question has been asked of operators since long before cable
 companies could offer internet service.  What happens if everyone in an
 area use their telephone (cellular or land line) at the same time?  A fast
 busy or recorded All circuits are busy message.  Over subscription is a
 fact of economics in virtually everything we do.  By this logic restaurants
 should be massively over built so that there is never a waiting line,
 highways should always be a speed limit ride, and all of these things would
 cost much more money than they do today.
 
 
 Scott Helms
 Vice President of Technology
 ZCorum
 (678) 507-5000
 
 http://twitter.com/kscotthelms
 
 
 
 On Sun, Apr 27, 2014 at 8:21 PM, Michael Thomas m...@mtcc.com wrote:
 
 Scott Helms wrote:
 
 Mark,
 
 Bandwidth use trends are actually increasingly asymmetical because of the
 popularity of OTT video.
 
 
 Until my other half decides to upload a video.
 
 Is it too much to ask for a bucket of bits that I can use in whichever
 direction happens
 to be needed at the moment?
 
 Mike

Re: Observations of an Internet Middleman (Level3) (was: RIP

Oh, I'm not proposing symmetrical connectivity at all. I'm just 
supporting the argument that in the context of this discussion I think 
it's silly for a residential ISP to purport themselves to be a neutral 
carrier of traffic and expect peering ratios to be symmetric when the 
overwhelming majority of what they're selling (and have been selling for 
over a decade) is asymmetric connectivity. Their traffic imbalance is, 
arguably, their own doing.


How residential ISPs recoup costs (or simply increase revenue/profit) is 
another question entirely. I think the most insightful comment in this 
discussion was made by Mr. Rick Astley (I assume a pseudonym), when he 
states that ISPs have several options to increase revenue A) Increase 
price of their product, B) Implement usage restrictions, or C) Charge 
someone else/Make someone else your customer. I think he successfully 
argues that option C may be the best. As we've seen, the wireless market 
in the US went for option B. We've yet to see where the wireline market 
will go.


Of course, the market would ideally keep ISPs' demands for 
revenue/profit in check and we'd all reach a satisfactory solution. One 
of the arguments, one I happen to support, in this thread is that there 
is not a free market for internet connectivity in many parts of the US. 
If there was, I believe Comcast would be focusing on how to provide a 
balance between the best product at the lowest cost and not on how they 
can monetize their paying customers in order to increase profits. I 
appreciate honesty; When a service provider advertises X Mbps Internet 
speeds, I expect they can deliver on their claims (to the whole 
Internet, and not just the portions of it they've decided). I understand 
congestion, overselling, etc. But choosing which portions of the 
internet work well and which don't is a lot more like censorship than 
service.


--Blake

Scott Helms wrote the following on 5/16/2014 12:39 PM:

Blake,

You're absolutely correct.  The world adapts to the reality that we 
find ourselves in via normal market mechanics.  The problem with 
proposing that connectivity for residential customers should be more 
symmetrical is that its expensive, which is why we as operators didn't 
roll it out that way to start.  We also don't see consumer demand for 
symmetrical connections and with the decline in peer to peer file 
sharing we've actually seen a decrease the ratio of used upstream 
bandwidth (though not a decrease in absolute terms).


I would like to deliver symmetrical bandwidth to all consumers just so 
those few customers who need it today would have lower bills but 
trying to justify that to our CFO without being able to point to an 
increase in revenue either because of more revenue per sub or more 
subs is a very tough task.  I don't believe my situation is uncommon.



Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 1:20 PM, Blake Hudson bl...@ispn.net 
mailto:bl...@ispn.net wrote:


Thanks for the insight Scott. I appreciate the experience and
point of view you're adding to this discussion (not just the
responses to me). While I might be playing the devil's advocate
here a bit, I think one could argue each of the points you've made
below.

I do feel that general usage patterns are a reflection of the
technologies that have traditionally been available to consumers.
New uses and applications would be available to overcome hurdles
if the technologies had developed to be symmetrical. I'm not
saying that the asymmetrical choice was a bad one, but it was not
without consequences. If residential ISPs sell asymmetric
connections for decades, how can the ISP expect that application
developers would not take this into account when developing
applications? I don't think my application would be very
successful if it required X Mbps and half of my market did not
meet this requirement. Of course content/service providers are
going to tailor their services based around their market.

--Blake

Scott Helms wrote the following on 5/16/2014 12:06 PM:

Blake,

I might agree with your premise if weren't for a couple of items.

1)  Very few consumers are walking around with a HD or 4K
camera today.

2)  Most consumers who want to share video wouldn't know how
to host it themselves, which isn't an insurmountable issue but
is a big barrier to entry especially given the number of
NAT'ed connections.  I think this is much more of a problem
than available bandwidth.

3)  Most consumers who want to share videos seem to be
satisfied with sharing via one of the cloud services whether
that be YouTube (which was created originally for that use),
Vimeo, or one of the other legions of

Re: Observations of an Internet Middleman (Level3)

Lazlo,

You're correct that some applications are being restricted, but AFAIK in
North America they are all being restricted for quite valid network
management reasons.  While back in the day I ran Sendmail and sometimes
qmail on my home connection I was also responsible with my mail server and
more importantly the world was different.  The threat from an open relay or
mail server with a compromise is much higher, in part because the speeds
are higher, but also because the attackers are more sophisticated and the
hardware the mail server is running on is much more powerful.  P2P is _not_
being blocked legally anywhere and if you believe that it is then you
should complain to the FCC in the US or the CRTC in Canada.  Running a DNS
or NTP server that's open to the Internet on a home connection should NOT
be allowed.  I'm sorry if you're one of the few people who can run those
services effectively and safely (just like SMTP) but the vast majority of
customers can't and in most cases they aren't running them intentionally.

I won't get into marketing, that's not what I do and I agree that unlimited
seems to mean something other than the way I understand it but that's no
different from unlimited telephone service, all you can eat buffets, or
just about anywhere else you can see the word unlimited or all in
marketing.  I'd also like to see much more competition in the market and
that's one the things I work to accomplish.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 2:38 PM, Laszlo Hanyecz las...@heliacal.net wrote:

 I'd just like to point out that a lot of people are in fact using their
 upstream capability, and the operators always throw a fit and try to cut
 off specific applications to force it back into the idle state.  For
 example P2P things like torrents and most recently the open NTP and DNS
 servers.  How about SMTP?  Not sure about you guys but my local broadband
 ISP has cut me off and told me that my 'unlimited internet' is in fact
 limited.  The reality is that those people who are not using it (99.8%?)
 are just being ripped off - paying for something they were told they need,
 thinking that it's there when they want it, then getting cut off when they
 actually try to use it.

 It's not like whining about it here will change anything, but the prices
 are severely distorted.  Triple play packages are designed to force people
 to pay for stuff they don't need or want - distorting the price of a
 service hoping to recover it elsewhere, then if the gamble doesn't pan out,
 the customer loses again.  The whole model is based on people buying stuff
 that they won't actually come to collect, so then you can sell it an
 infinite number of times.  The people who do try to collect what was sold
 to them literally end up getting called names and cut off - terms like
 excessive bandwidth user and network abuser are used to describe paying
 customers.  With regard to the peering disputes, it's hardly surprising
 that their business partners are treated with the same attitude as their
 customers.  Besides, if you cut off the customers and peers who are causing
 that saturation, then the existing peering links can support an infinite
 number of idle subscribers.  The next phase is usage-based-billing which is
 kind of like having to pay a fine for using it, so they can artificially
 push the price point lower and hopefully get some more idle customers.
  That will help get the demand down and keep the infrastructure nice and
 idle.  When you're paying for every cat video maybe you realize you can
 live without it instead.

 Everyone has been trained so well, they don't even flinch anymore when
 they hear about over subscription, and they apologize for the people who
 are doing it to them.  The restaurant analogy is incorrect - you can go to
 the restaurant next door if a place is busy, thus they have pressure to
 increase their capacity if they want to sell more meals.  With broadband
 you can't go anywhere else, (for most people) there's only one restaurant,
 and there's a week long waiting list.  If you don't like it, you're
 probably an abuser or excessive eater anyway.

 -Laszlo


 On May 16, 2014, at 5:34 PM, Scott Helms khe...@zcorum.com wrote:

  Michael,
 
  No, its not too much to ask and any end user who has that kind of
  requirement can order a business service to get symmetrical service but
 the
  reality is that symmetrical service costs more and the vast majority of
  customers don't use the upstream capacity they have today.  I have
 personal
  insight into about half a million devices and the percentage of people
 who
  bump up against their upstream rate is less than 0.2%.  I have the
 ability
  to get data on another 10 million and the last time I checked their rates
  were similar.
 
  This kind of question has been asked of operators since long

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Fri, May 16, 2014 at 2:47 PM, Blake Hudson bl...@ispn.net wrote:
 in the context of this discussion I think it's silly for a residential ISP
 to purport themselves to be a neutral carrier of traffic and expect peering
 ratios to be symmetric

is 'symmetric traffic ratios' even relevant though? Peering is about
offsetting costs, right? it might not be important that the ratio be
1:1 or 2:1... or even 10:1, if it's going to cost you 20x to get the
traffic over longer/transit/etc paths... or if you have to build into
some horrific location(s) to access the content in question.

Harping on symmetric ratios seems very 1990... and not particularly
germaine to the conversation at hand.

Re: Observations of an Internet Middleman (Level3) (was: RIP

Blake,

I'm not sure what the relationship between what an access network sells has
to do with how their peering is done.  I realize that everyone's favorite
target is Comcast right now, but would anyone bat an eye over ATT making
the same requirement since they have much more in the way of transit
traffic?  I don't think anyone forced Level 3 into their peering agreement
with Comcast and it was (roughly) symmetrical for years before Level 3 was
contracted by Netflix.  Shouldn't Level 3 gone to Comcast and told them
they needed to change their peering or get a different contract?  Why was
Cogent able to maintain (roughly) symmetrical traffic with Comcast when
they were the primary path for Netflix to Comcast users?

Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 2:47 PM, Blake Hudson bl...@ispn.net wrote:

 Oh, I'm not proposing symmetrical connectivity at all. I'm just supporting
 the argument that in the context of this discussion I think it's silly for
 a residential ISP to purport themselves to be a neutral carrier of traffic
 and expect peering ratios to be symmetric when the overwhelming majority of
 what they're selling (and have been selling for over a decade) is
 asymmetric connectivity. Their traffic imbalance is, arguably, their own
 doing.

 How residential ISPs recoup costs (or simply increase revenue/profit) is
 another question entirely. I think the most insightful comment in this
 discussion was made by Mr. Rick Astley (I assume a pseudonym), when he
 states that ISPs have several options to increase revenue A) Increase price
 of their product, B) Implement usage restrictions, or C) Charge someone
 else/Make someone else your customer. I think he successfully argues that
 option C may be the best. As we've seen, the wireless market in the US went
 for option B. We've yet to see where the wireline market will go.

 Of course, the market would ideally keep ISPs' demands for revenue/profit
 in check and we'd all reach a satisfactory solution. One of the arguments,
 one I happen to support, in this thread is that there is not a free market
 for internet connectivity in many parts of the US. If there was, I believe
 Comcast would be focusing on how to provide a balance between the best
 product at the lowest cost and not on how they can monetize their paying
 customers in order to increase profits. I appreciate honesty; When a
 service provider advertises X Mbps Internet speeds, I expect they can
 deliver on their claims (to the whole Internet, and not just the portions
 of it they've decided). I understand congestion, overselling, etc. But
 choosing which portions of the internet work well and which don't is a lot
 more like censorship than service.

 --Blake

Re: Observations of an Internet Middleman (Level3) (was: RIP



Christopher Morrow wrote the following on 5/16/2014 1:52 PM:

On Fri, May 16, 2014 at 2:47 PM, Blake Hudson bl...@ispn.net wrote:

in the context of this discussion I think it's silly for a residential ISP
to purport themselves to be a neutral carrier of traffic and expect peering
ratios to be symmetric

is 'symmetric traffic ratios' even relevant though? Peering is about
offsetting costs, right? it might not be important that the ratio be
1:1 or 2:1... or even 10:1, if it's going to cost you 20x to get the
traffic over longer/transit/etc paths... or if you have to build into
some horrific location(s) to access the content in question.

Harping on symmetric ratios seems very 1990... and not particularly
germaine to the conversation at hand.
I agree about the term being passe ...and that it never applied to ISPs 
...and that peering is about cost reduction, reliability, and 
performance. It seems to me that many CDNs or content providers want to 
setup peering relationships and are willing to do so at a cost to them 
in order to bypass the internet middle men. But I mention traffic 
ratios because some folks in this discussion seem to be using it as 
justification for not peering. But hey, why peer at little or no cost if 
they can instead hold out and possibly peer at a negative cost?


--Blake

Re: Observations of an Internet Middleman (Level3) (was: RIP

2014-05-16 Thread James R Cutler


All this talk about symmetry and asymmetry is interesting.  

Has anyone actually quantified how much congestion is due to buffer bloat which 
is, in turn, exacerbated by asymmetric connections?


James R. Cutler
james.cut...@consultant.com
PGP keys at http://pgp.mit.edu





signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Observations of an Internet Middleman (Level3) (was: RIP

2014-05-16 Thread Matthew Petach

On Fri, May 16, 2014 at 11:52 AM, Christopher Morrow 
morrowc.li...@gmail.com wrote:

 On Fri, May 16, 2014 at 2:47 PM, Blake Hudson bl...@ispn.net wrote:
  in the context of this discussion I think it's silly for a residential
 ISP
  to purport themselves to be a neutral carrier of traffic and expect
 peering
  ratios to be symmetric

 is 'symmetric traffic ratios' even relevant though? Peering is about
 offsetting costs, right? it might not be important that the ratio be
 1:1 or 2:1... or even 10:1, if it's going to cost you 20x to get the
 traffic over longer/transit/etc paths... or if you have to build into
 some horrific location(s) to access the content in question.

 Harping on symmetric ratios seems very 1990... and not particularly
 germaine to the conversation at hand.


Traffic asymmetry across peering connections
was what lit the fuse on this whole powder keg,
if I understand correctly; at the point the traffic
went asymmetric, the refusals to augment
capacity kicked in, and congestion became
a problem.

I've seen the same thing; pretty much every
rejection is based on ratio issues, even when
offering to cold-potato haul the traffic to the
home market for the users.

If the refusals hinged on any other clause
of the peering requirements, you'd be right;
but at the moment, that's the flag networks
are waving around as their speedbump-du-jour.
So, it may be very 1990, but unfortunately
that seems to be the year many people in
the industry are mentally stuck in.  :(

Matt

Re: Observations of an Internet Middleman (Level3) (was: RIP

2014-05-16 Thread Matthew Petach

On Fri, May 16, 2014 at 12:14 PM, James R Cutler 
james.cut...@consultant.com wrote:


 All this talk about symmetry and asymmetry is interesting.

 Has anyone actually quantified how much congestion is due to buffer bloat
 which is, in turn, exacerbated by asymmetric connections?


 James R. Cutler
 james.cut...@consultant.com
 PGP keys at http://pgp.mit.edu


I think you might have the cart before the horse.

If there's no congestion on a peering link,
buffering doesn't come into play, at least
not within the transport infrastructure.

We're not talking congestion on the last mile
side, we're looking at congestion on the
interconnect links between networks,
typically 10G or 100G ports.  Unless
you're running those links near or at
capacity, buffering should be a complete
non-issue.  And if you're running those
links at capacity, then the congestion
is due to too much traffic, period, not
to the size of buffers involved on either
side of the link.  ^_^;

Thanks!

Matt

Re: Observations of an Internet Middleman (Level3) (was: RIP

Matthew,

There is a difference between what should be philosophically and what
happened with Level 3 which is a contractual issue.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 3:15 PM, Matthew Petach mpet...@netflight.comwrote:

 On Fri, May 16, 2014 at 11:52 AM, Christopher Morrow 
 morrowc.li...@gmail.com wrote:

  On Fri, May 16, 2014 at 2:47 PM, Blake Hudson bl...@ispn.net wrote:
   in the context of this discussion I think it's silly for a residential
  ISP
   to purport themselves to be a neutral carrier of traffic and expect
  peering
   ratios to be symmetric
 
  is 'symmetric traffic ratios' even relevant though? Peering is about
  offsetting costs, right? it might not be important that the ratio be
  1:1 or 2:1... or even 10:1, if it's going to cost you 20x to get the
  traffic over longer/transit/etc paths... or if you have to build into
  some horrific location(s) to access the content in question.
 
  Harping on symmetric ratios seems very 1990... and not particularly
  germaine to the conversation at hand.
 
 
 Traffic asymmetry across peering connections
 was what lit the fuse on this whole powder keg,
 if I understand correctly; at the point the traffic
 went asymmetric, the refusals to augment
 capacity kicked in, and congestion became
 a problem.

 I've seen the same thing; pretty much every
 rejection is based on ratio issues, even when
 offering to cold-potato haul the traffic to the
 home market for the users.

 If the refusals hinged on any other clause
 of the peering requirements, you'd be right;
 but at the moment, that's the flag networks
 are waving around as their speedbump-du-jour.
 So, it may be very 1990, but unfortunately
 that seems to be the year many people in
 the industry are mentally stuck in.  :(

 Matt

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Friday, May 16, 2014 05:35:39 PM Jay Ashworth wrote:

 Could you expand a bit, Mark on Social media forces the
 use of symmetric bandwidth?  Which social media
 platform is it that you think has a) symmetrical flows
 that b) are big enough to figure into transit symmetry?

What we saw with FTTH deployments is that customers uploaded 
more videos and photos to Youtube, Facebook, MySpace, e.t.c. 
They didn't do this on ADSL as much (it's too frustrating).

When that caught on, customers started buying online backup 
services - synchronizing backups of their home or office 
computers to remote backup infrastructure. Again, they never 
did this with ADSL.

What we learned: don't take it for granted that you will 
always know what your customers (or the content providers 
who serve them) will do with the bandwidth. If they have it, 
expect the worst, and plan for it as best you can.

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Fri, May 16, 2014 at 3:15 PM, Matthew Petach mpet...@netflight.com wrote:



 On Fri, May 16, 2014 at 11:52 AM, Christopher Morrow
 morrowc.li...@gmail.com wrote:

 On Fri, May 16, 2014 at 2:47 PM, Blake Hudson bl...@ispn.net wrote:
  in the context of this discussion I think it's silly for a residential
  ISP
  to purport themselves to be a neutral carrier of traffic and expect
  peering
  ratios to be symmetric

 is 'symmetric traffic ratios' even relevant though? Peering is about
 offsetting costs, right? it might not be important that the ratio be
 1:1 or 2:1... or even 10:1, if it's going to cost you 20x to get the
 traffic over longer/transit/etc paths... or if you have to build into
 some horrific location(s) to access the content in question.

 Harping on symmetric ratios seems very 1990... and not particularly
 germaine to the conversation at hand.


 Traffic asymmetry across peering connections
 was what lit the fuse on this whole powder keg,
 if I understand correctly; at the point the traffic
 went asymmetric, the refusals to augment
 capacity kicked in, and congestion became
 a problem.

Is it that? or is it that planning at some ISP pair had a '6 months to
upgrade' regularly penciled in, then 'all of a sudden' their links
were filling up faster than every 6months and... now they are 1x or 2x
upgrade cycles behind?

I imagine that up to a point upgrading a router that does only
'peering' (SFP) is 'easy', but at some step function of upgrades on
the edge ports you need to provision more backhaul and more core and
probably upgrade the link types and the chassis and ...

At some ISPs this process involves more than 1 dude/group. So
coordination and budget issues and scheduling ... become a bit harder.
Adjusting to the new reality of 'you need to plan for pipe filling
more often, increase upgrade cycle crank speed!' seems like at least
one problem, to me at least.

It's really hard to tell what's upsetting people about this whole topic :(
There's a mix of 'my access link blows' to 'isps should peer better
and for freer' and a bunch of other stuff all mixed in the middle :(

 I've seen the same thing; pretty much every
 rejection is based on ratio issues, even when
 offering to cold-potato haul the traffic to the
 home market for the users.

yes, welp... it's often rough to get folk who want to think in terms
of apples to suddenly thing in terms of the new best fruit 'acai
berry'. Especially at large and entrenched organizations.

 If the refusals hinged on any other clause
 of the peering requirements, you'd be right;
 but at the moment, that's the flag networks
 are waving around as their speedbump-du-jour.

sure, it's also super easy for them to do this, see entrenched org
comment above.
it seems to me that the point of peering is not stalin voice'ratio
or bust'/stalin voice but 'mutual benefit'. If a skewed ratio of
100:1 in a local market still is cheaper than 'backhaul that traffic
from LHR to SFO' there's mutual benefit and a reason to peer.

I understand that this is a bit of a rosy landscape I'm painting, but...

 So, it may be very 1990, but unfortunately
 that seems to be the year many people in
 the industry are mentally stuck in.  :(

oh, entrenched. I see.

thanks!
-chris

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Fri, May 16, 2014 at 3:11 PM, Blake Hudson bl...@ispn.net wrote:

 Christopher Morrow wrote the following on 5/16/2014 1:52 PM:

 On Fri, May 16, 2014 at 2:47 PM, Blake Hudson bl...@ispn.net wrote:

 in the context of this discussion I think it's silly for a residential
 ISP
 to purport themselves to be a neutral carrier of traffic and expect
 peering
 ratios to be symmetric

 is 'symmetric traffic ratios' even relevant though? Peering is about
 offsetting costs, right? it might not be important that the ratio be
 1:1 or 2:1... or even 10:1, if it's going to cost you 20x to get the
 traffic over longer/transit/etc paths... or if you have to build into
 some horrific location(s) to access the content in question.

 Harping on symmetric ratios seems very 1990... and not particularly
 germaine to the conversation at hand.

 I agree about the term being passe ...and that it never applied to ISPs
 ...and that peering is about cost reduction, reliability, and performance.

ok.

 It seems to me that many CDNs or content providers want to setup peering
 relationships and are willing to do so at a cost to them in order to bypass
 the internet middle men. But I mention traffic ratios because some folks

'the internet middle men' - is really, it seems to me, 'people I have
no business relationship with'. There's also no way to control the
capacity planning process with these middle-men, right? Some AS in the
middle of my 3-AS-way conversation isn't someone I can capacity plan
with :(

-chris

 in this discussion seem to be using it as justification for not peering. But
 hey, why peer at little or no cost if they can instead hold out and possibly
 peer at a negative cost?

 --Blake

Re: Observations of an Internet Middleman (Level3) (was: RIP

On Friday, May 16, 2014 05:45:06 PM Scott Helms wrote:

 Bandwidth use trends are actually increasingly
 asymmetical because of the popularity of OTT video.
 
 Social media, even with video uploading, simply doesn't
 generate that much traffic per session.

Our experience showed that there is a direct co-relation 
between the lack of traffic in the upstream direction and 
poor upload bandwidth (primarily, due to asymmetric tech. 
such as ADSL), e.g., because of the ADSL I have at home 
(512Kbps up, 4Mbps down), I generally do not send very large 
e-mails when working from home; nor do I use my laptop for 
remote router/switch updates as the software images are a 
nightmare to upload.

And yes, there is a larger proportion of downstream traffic 
than there is upstream traffic pretty much most of the time 
(even with symmetric links). However, with symmetry, 
upstream traffic will increase significantly as customers 
realize it is now available.

One of the use-cases we thought about when deploying an FTTH 
backbone was having remote PVR's. So rather than record and 
save linear Tv programming on the STB, record and save it in 
the network. This could only be done with symmetric 
bandwidth.

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Observations of an Internet Middleman (Level3) (was: RIP

Mark,

 I don't think that anyone disputes that when you improve the upstream you
do get an uptick in usage in that direction.  What I take issue with is the
notion that the upstream is anything like downstream even when the capacity
is there.  Upstream on ADSL is horribad, especially the first generations
(g.lite and g.dmt).


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 3:25 PM, Mark Tinka mark.ti...@seacom.mu wrote:

 On Friday, May 16, 2014 05:35:39 PM Jay Ashworth wrote:

  Could you expand a bit, Mark on Social media forces the
  use of symmetric bandwidth?  Which social media
  platform is it that you think has a) symmetrical flows
  that b) are big enough to figure into transit symmetry?

 What we saw with FTTH deployments is that customers uploaded
 more videos and photos to Youtube, Facebook, MySpace, e.t.c.
 They didn't do this on ADSL as much (it's too frustrating).

 When that caught on, customers started buying online backup
 services - synchronizing backups of their home or office
 computers to remote backup infrastructure. Again, they never
 did this with ADSL.

 What we learned: don't take it for granted that you will
 always know what your customers (or the content providers
 who serve them) will do with the bandwidth. If they have it,
 expect the worst, and plan for it as best you can.

 Mark.

Re: Observations of an Internet Middleman (Level3)


Scott Helms wrote:

Michael,

No, its not too much to ask and any end user who has that kind of 
requirement can order a business service to get symmetrical service but 
the reality is that symmetrical service costs more and the vast majority 
of customers don't use the upstream capacity they have today.  I have 
personal insight into about half a million devices and the percentage of 
people who bump up against their upstream rate is less than 0.2%.  I 
have the ability to get data on another 10 million and the last time I 
checked their rates were similar.


I've just been on the losing end of yet another piece of why crappy upstream
bandwidth sucks: Mavericks seems to have decided that my other half's imovie
library really, really ought to be uploaded to iCloud (without asking, ftw).

I can and should be pissed at Apple for doing such a wrongheaded thing, but
the fact is that my upstream bandwidth was saturated for hours and days and it
was extremely difficult to figure out why. I doubt I'm alone.

Better upstream bandwidth would have at least made the pain period shorter.

Mike

Re: Observations of an Internet Middleman (Level3)


Mark Tinka wrote:

One of the use-cases we thought about when deploying an FTTH 
backbone was having remote PVR's. So rather than record and 
save linear Tv programming on the STB, record and save it in 
the network. This could only be done with symmetric 
bandwidth.




Isn't this already the case with Dishtv and their partnership with Sling?
I'm pretty sure it's streaming it direct from my home dvr.

Mike

Re: Observations of an Internet Middleman (Level3)

Mike,

In my experience you're not alone, just in a really tiny group.  As I said
I have direct eyeballs on ~500k devices and the ability to see another 10
million anytime I want and the percentage of people who cap their upstream
in both of those sample groups for more than 15 minutes (over the last 3
years) is about 0.2%.  Interestingly if a customer does it once they have
about a 70% chance of doing it regularly.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 3:46 PM, Michael Thomas m...@mtcc.com wrote:

 Scott Helms wrote:

 Michael,

 No, its not too much to ask and any end user who has that kind of
 requirement can order a business service to get symmetrical service but the
 reality is that symmetrical service costs more and the vast majority of
 customers don't use the upstream capacity they have today.  I have personal
 insight into about half a million devices and the percentage of people who
 bump up against their upstream rate is less than 0.2%.  I have the ability
 to get data on another 10 million and the last time I checked their rates
 were similar.


 I've just been on the losing end of yet another piece of why crappy
 upstream
 bandwidth sucks: Mavericks seems to have decided that my other half's
 imovie
 library really, really ought to be uploaded to iCloud (without asking,
 ftw).

 I can and should be pissed at Apple for doing such a wrongheaded thing, but
 the fact is that my upstream bandwidth was saturated for hours and days
 and it
 was extremely difficult to figure out why. I doubt I'm alone.

 Better upstream bandwidth would have at least made the pain period shorter.

 Mike

Re: Observations of an Internet Middleman (Level3)


Scott Helms wrote:

Mike,

In my experience you're not alone, just in a really tiny group.  As I 
said I have direct eyeballs on ~500k devices and the ability to see 
another 10 million anytime I want and the percentage of people who cap 
their upstream in both of those sample groups for more than 15 minutes 
(over the last 3 years) is about 0.2%.  Interestingly if a customer does 
it once they have about a 70% chance of doing it regularly.


Well, given Sling, Dropbox, iCloud, pervasive video calls (you have heard about 
webrtc, yes?
24/7 babycams!), youtube, etc, etc, I won't be a tiny group for long.

Mike

Re: Observations of an Internet Middleman (Level3)

I think you will, all of those things have been around for a long time
(well, except for pervasive video calls, which I think is vapor) and none
generate the kind of traffic it takes to congest a decent link.  Most of
the DOCSIS systems I've worked with are running at least 6 mbps upstreams
and many are well into the double digits.  My current connection (tested
this morning) is about 22 mbps.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 4:06 PM, Michael Thomas m...@mtcc.com wrote:

 Scott Helms wrote:

 Mike,

 In my experience you're not alone, just in a really tiny group.  As I
 said I have direct eyeballs on ~500k devices and the ability to see another
 10 million anytime I want and the percentage of people who cap their
 upstream in both of those sample groups for more than 15 minutes (over the
 last 3 years) is about 0.2%.  Interestingly if a customer does it once they
 have about a 70% chance of doing it regularly.


 Well, given Sling, Dropbox, iCloud, pervasive video calls (you have heard
 about webrtc, yes?
 24/7 babycams!), youtube, etc, etc, I won't be a tiny group for long.

 Mike

Re: Observations of an Internet Middleman (Level3)


Scott Helms wrote:
I think you will, all of those things have been around for a long time 
(well, except for pervasive video calls, which I think is vapor) and 
none generate the kind of traffic it takes to congest a decent link. 
 Most of the DOCSIS systems I've worked with are running at least 6 mbps 
upstreams and many are well into the double digits.  My current 
connection (tested this morning) is about 22 mbps.


Um, no it's not vapor. Webrtc is quite real, and the barrier to implementation
for any random web site is weeks, not years as was the case before.

I just saw this that you wrote:

1)  Very few consumers are walking around with a HD or 4K camera today.

In the US, we just surpassed 1/2 of the population who have that capability, 
iirc. They
call them phones nowadays.

Mike



Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms
 



On Fri, May 16, 2014 at 4:06 PM, Michael Thomas m...@mtcc.com 
mailto:m...@mtcc.com wrote:


Scott Helms wrote:

Mike,

In my experience you're not alone, just in a really tiny group.
 As I said I have direct eyeballs on ~500k devices and the
ability to see another 10 million anytime I want and the
percentage of people who cap their upstream in both of those
sample groups for more than 15 minutes (over the last 3 years)
is about 0.2%.  Interestingly if a customer does it once they
have about a 70% chance of doing it regularly.


Well, given Sling, Dropbox, iCloud, pervasive video calls (you have
heard about webrtc, yes?
24/7 babycams!), youtube, etc, etc, I won't be a tiny group for long.

Mike

Re: Observations of an Internet Middleman (Level3)

Michael,

I didn't claim Webrtc is vapor, I claim that pervasive video calling is
vapor.  Further, even if that prediction is wrong pervasive video calling
isn't enough even if 100% of users adopt it to swing the need for
symmetrical bandwidth.  An average Skype/Google Hangout/Apple is less than
400 kbps at peak and averages something like 150 kbps.

http://www.digitalsociety.org/2010/08/iphone-facetime-bandwidth-gets-measured/




Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Fri, May 16, 2014 at 4:22 PM, Michael Thomas m...@mtcc.com wrote:

 Scott Helms wrote:

 I think you will, all of those things have been around for a long time
 (well, except for pervasive video calls, which I think is vapor) and none
 generate the kind of traffic it takes to congest a decent link.  Most of
 the DOCSIS systems I've worked with are running at least 6 mbps upstreams
 and many are well into the double digits.  My current connection (tested
 this morning) is about 22 mbps.


 Um, no it's not vapor. Webrtc is quite real, and the barrier to
 implementation
 for any random web site is weeks, not years as was the case before.

 I just saw this that you wrote:

 1)  Very few consumers are walking around with a HD or 4K camera
 today.

 In the US, we just surpassed 1/2 of the population who have that
 capability, iirc. They
 call them phones nowadays.

 Mike


 Scott Helms
 Vice President of Technology
 ZCorum
 (678) 507-5000
 
 http://twitter.com/kscotthelms
 

 On Fri, May 16, 2014 at 4:06 PM, Michael Thomas m...@mtcc.com mailto:
 m...@mtcc.com wrote:

 Scott Helms wrote:

 Mike,

 In my experience you're not alone, just in a really tiny group.
  As I said I have direct eyeballs on ~500k devices and the
 ability to see another 10 million anytime I want and the
 percentage of people who cap their upstream in both of those
 sample groups for more than 15 minutes (over the last 3 years)
 is about 0.2%.  Interestingly if a customer does it once they
 have about a 70% chance of doing it regularly.


 Well, given Sling, Dropbox, iCloud, pervasive video calls (you have
 heard about webrtc, yes?
 24/7 babycams!), youtube, etc, etc, I won't be a tiny group for
 long.

 Mike

Re: Observations of an Internet Middleman (Level3)

2014-05-16 Thread Jared Mauch


On May 16, 2014, at 4:22 PM, Michael Thomas m...@mtcc.com wrote:

 In the US, we just surpassed 1/2 of the population who have that capability, 
 iirc. They
 call them phones nowadays.

Many of them have native IPv6 as well, this also hasn't gotten significant 
number of legacy/incumbents to deploy yet either.  It seems Facebook/LTE are 
the killer apps for v6.

http://www.internetsociety.org/deploy360/wp-content/uploads/2014/04/WorldIPv6Congress-IPv6_LH-v2.pdf

Like all things there are leaders and followers and the long-tail.

- Jared

Re: Observations of an Internet Middleman (Level3) (was: RIP

2014-05-16 Thread Ca By

On May 16, 2014 12:21 PM, Matthew Petach mpet...@netflight.com wrote:

 On Fri, May 16, 2014 at 11:52 AM, Christopher Morrow 
 morrowc.li...@gmail.com wrote:

  On Fri, May 16, 2014 at 2:47 PM, Blake Hudson bl...@ispn.net wrote:
   in the context of this discussion I think it's silly for a residential
  ISP
   to purport themselves to be a neutral carrier of traffic and expect
  peering
   ratios to be symmetric
 
  is 'symmetric traffic ratios' even relevant though? Peering is about
  offsetting costs, right? it might not be important that the ratio be
  1:1 or 2:1... or even 10:1, if it's going to cost you 20x to get the
  traffic over longer/transit/etc paths... or if you have to build into
  some horrific location(s) to access the content in question.
 
  Harping on symmetric ratios seems very 1990... and not particularly
  germaine to the conversation at hand.
 
 
 Traffic asymmetry across peering connections
 was what lit the fuse on this whole powder keg,
 if I understand correctly; at the point the traffic
 went asymmetric, the refusals to augment
 capacity kicked in, and congestion became
 a problem.


What lit this powder keg?:

conspiracy theory

Netflix bought transit from cogent and expected it to work. C'mon.  This
happens every month on this list and every month people tell others not to
rely on cogent. Right? Netflix is smart, they know cogent is willing to
burn down their network and blow up their customers for 15 minutes of fame
$0.03 a meg.

This makes me think the whole thing is a net neutrality strawman.

They set the stage and all the players played their part.

Now, what will be the result?  I expect some concession from the
comcast/twc deal.  They made a big deal about net neutrality  / netflix /
strawman so they can trump up a meaningful concession to allow the
merger.

/conspiracy

 I've seen the same thing; pretty much every
 rejection is based on ratio issues, even when
 offering to cold-potato haul the traffic to the
 home market for the users.

 If the refusals hinged on any other clause
 of the peering requirements, you'd be right;
 but at the moment, that's the flag networks
 are waving around as their speedbump-du-jour.
 So, it may be very 1990, but unfortunately
 that seems to be the year many people in
 the industry are mentally stuck in.  :(

 Matt

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

Oh, please do explicate on how this is inaccurate…

Owen

On May 14, 2014, at 2:14 PM, McElearney, Kevin 
kevin_mcelear...@cable.comcast.com wrote:

 Respectfully, this is a highly inaccurate sound bite
 
 - Kevin
 
 215-313-1083
 
 On May 14, 2014, at 3:05 PM, Owen DeLong o...@delong.com wrote:
 
 Yes, the more accurate statement would be aggressively seeking new
 ways to monetize the existing infrastructure without investing in upgrades
 or additional buildout any more than absolutely necessary.
 
 Owen
 
 On May 14, 2014, at 8:02 AM, Hugo Slabbert h...@slabnet.com wrote:
 
 
 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin).
 
 Sure they are (seeking new sources of revenue).  They're not necessarily
 creating new products or services, i.e. actually adding any value, but they
 are finding ways to extract additional revenue from the same pipes, e.g.
 through paid peering with content providers.
 
 I'm not endorsing this; just pointing out that you two are actually in
 agreement here.
 
 --
 Hugo
 
 
 On Wed, May 14, 2014 at 7:23 AM, char...@thefnf.org wrote:
 
 On 2014-05-14 02:04, Jean-Francois Mezei wrote:
 
 On 14-05-13 22:50, Daniel Staal wrote:
 
 They have the money.  They have the ability to get more money.  *They see
 no reason to spend money making customers happy.*  They can make more
 profit without it.
 
 There is the issue of control over the market. But also the pressure
 from shareholders for continued growth.
 
 
 Yes. That is true. Except that it's not.
 
 How do service providers grow? Let's explore that:
 
 What is growth for a transit provider?
 
 More (new) access network(s) (connections).
 More bandwidth across backbone pipes.
 
 
 What is growth for access network?
 More subscribers.
 
 Except that the incumbent carriers have shown they have no interest in
 providing decent bandwidth to anywhere but the most profitable rate
 centers. I'd say about 2/3 of the USA is served with quite terrible access.
 
 
 
 
 The problem with the internet is that while it had promises of wild
 growth in the 90s and 00s, once penetration reaches a certain level,
 growth stabilizes.
 
 Penetration is ABYSMAL sir. Huge swaths of underserved americans exist.
 
 
 
 When you combine this with threath to large incumbents's media and media
 distribution endeavours by the likes of Netflix (and cat videos on
 Youtube), large incumbents start thinking about how they will be able to
 continue to grow revenus/profits when customers will shift spending to
 vspecialty channels/cableTV to Netflix and customer growth will not
 compensate.
 
 Except they aren't. Even in the most profitable rate centers, they've
 declined to really invest in the networks. They aren't a real business. You
 have to remember that. They have regulatory capture, natural/defacto
 monopoly etc etc. They don't operate in the real world of
 risk/reward/profit/loss/uncertainty like any other real business has to.
 
 
 
 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin). They did
 a bit of an advertising campaign of smart home offerings, but that seems
 to have never grown beyond a pilot.
 
 
 
 The current trend is to if you can't fight them, jon them where
 cablecos start to include the Netflix app into their proprietary set-top
 boxes. The idea is that you at least make the customer continue to use
 your box and your remote control which makes it easier for them to
 switch between netflix and legacy TV.
 True. I don't know why one of the cablecos hasn't licensed roku, added
 cable card and made that available as a hip/cool set top box offering and
 charge another 10.00 a month on top of the standard dvr rental.
 
 
 
 Would be interesting to see if those cable companies that are agreeing
 to add the Netflix app onto their proprietary STBs also  play peering
 capacity games to degrade the service or not.
 
 So how is the content delivered? Is it over the internet? Or is it over
 the cable plant, from cable headends?

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

I don’t disagree.  However, given the choice between Comcast and broadband 
services in NL, Chatanooga, or Seoul, just to name a few, Comcast loses badly.

Choosing between Comcast and a legacy Telco is like choosing between 
legionnaire’s disease and SARS.

Owen

On May 14, 2014, at 5:15 PM, Jared Mauch ja...@puck.nether.net wrote:

 Owen,
 
 I've seen a vast difference between Comcast and others in the marketplace.  
 Right now, if I had the choice between Comcast and a legacy telco, I would 
 pick Comcast hands-down for:
 
 a) performance
 b) IPv6 support
 c) willingness to work on issues
 
 - Jared
 
 On May 14, 2014, at 5:14 PM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.com wrote:
 
 Respectfully, this is a highly inaccurate sound bite
 
- Kevin
 
 215-313-1083
 
 On May 14, 2014, at 3:05 PM, Owen DeLong o...@delong.com wrote:
 
 Yes, the more accurate statement would be aggressively seeking new
 ways to monetize the existing infrastructure without investing in upgrades
 or additional buildout any more than absolutely necessary.
 
 Owen
 
 On May 14, 2014, at 8:02 AM, Hugo Slabbert h...@slabnet.com wrote:
 
 
 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin).
 
 Sure they are (seeking new sources of revenue).  They're not necessarily
 creating new products or services, i.e. actually adding any value, but they
 are finding ways to extract additional revenue from the same pipes, e.g.
 through paid peering with content providers.
 
 I'm not endorsing this; just pointing out that you two are actually in
 agreement here.
 
 --
 Hugo
 
 
 On Wed, May 14, 2014 at 7:23 AM, char...@thefnf.org wrote:
 
 On 2014-05-14 02:04, Jean-Francois Mezei wrote:
 
 On 14-05-13 22:50, Daniel Staal wrote:
 
 They have the money.  They have the ability to get more money.  *They see
 no reason to spend money making customers happy.*  They can make more
 profit without it.
 
 There is the issue of control over the market. But also the pressure
 from shareholders for continued growth.
 
 
 Yes. That is true. Except that it's not.
 
 How do service providers grow? Let's explore that:
 
 What is growth for a transit provider?
 
 More (new) access network(s) (connections).
 More bandwidth across backbone pipes.
 
 
 What is growth for access network?
 More subscribers.
 
 Except that the incumbent carriers have shown they have no interest in
 providing decent bandwidth to anywhere but the most profitable rate
 centers. I'd say about 2/3 of the USA is served with quite terrible 
 access.
 
 
 
 
 The problem with the internet is that while it had promises of wild
 growth in the 90s and 00s, once penetration reaches a certain level,
 growth stabilizes.
 
 Penetration is ABYSMAL sir. Huge swaths of underserved americans exist.
 
 
 
 When you combine this with threath to large incumbents's media and media
 distribution endeavours by the likes of Netflix (and cat videos on
 Youtube), large incumbents start thinking about how they will be able to
 continue to grow revenus/profits when customers will shift spending to
 vspecialty channels/cableTV to Netflix and customer growth will not
 compensate.
 
 Except they aren't. Even in the most profitable rate centers, they've
 declined to really invest in the networks. They aren't a real business. 
 You
 have to remember that. They have regulatory capture, natural/defacto
 monopoly etc etc. They don't operate in the real world of
 risk/reward/profit/loss/uncertainty like any other real business has to.
 
 
 
 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin). They 
 did
 a bit of an advertising campaign of smart home offerings, but that seems
 to have never grown beyond a pilot.
 
 
 
 The current trend is to if you can't fight them, jon them where
 cablecos start to include the Netflix app into their proprietary set-top
 boxes. The idea is that you at least make the customer continue to use
 your box and your remote control which makes it easier for them to
 switch between netflix and legacy TV.
 True. I don't know why one of the cablecos hasn't licensed roku, added
 cable card and made that available as a hip/cool set top box offering 
 and
 charge another 10.00 a month on top of the standard dvr rental.
 
 
 
 Would be interesting to see if those cable companies that are agreeing
 to add the Netflix app onto their proprietary STBs also  play peering
 capacity games to degrade the service or not.
 
 So how is the content delivered? Is it over the internet? Or is it over
 the

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

Having an actual free market would require having competition. So long as we 
have monopoly layer 1 providers being allowed to use that monopoly as leverage 
for higher layer service monopolies, (or oligopolies), an actual free market is 
virtually impossible.

The result of deregulating the current environment would only be more pain and 
cost to the consumer than we currently have with no improvement in speeds or 
capabilities and no additional innovation.

Owen

On May 14, 2014, at 6:05 PM, Matt Palmer mpal...@hezmatt.org wrote:

 On Wed, May 14, 2014 at 07:01:36PM -0500, Larry Sheldon wrote:
 Maybe it is time to try a free market.
 
 Can't do that, it would be UnAmerican!
 
 - Matt
 
 -- 
 I can only guess that the designer of the things had a major Toilet Duck
 habit and had managed to score a couple of industrial-sized bottles of the
 stuff the night before.
   -- Tanuki

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

Upgrades/buildout are happening every day.  They are continuous to keep ahead 
of demand and publicly measured by SamKnows (FCC measuring broadband), Akamai, 
Ookla, etc

What is not well known is that Comcast has been an existing commercial transit 
business for 15+ years (with over 8000 commercial fiber customers).  Comcast 
also has over 40 balanced peers with plenty of capacity, and some of the 
largest Internet companies as customers.

  - Kevin

215-313-1083

 On May 15, 2014, at 10:19 AM, Owen DeLong o...@delong.com wrote:
 
 Oh, please do explicate on how this is inaccurate…
 
 Owen
 
 On May 14, 2014, at 2:14 PM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.com wrote:
 
 Respectfully, this is a highly inaccurate sound bite
 
- Kevin
 
 215-313-1083
 
 On May 14, 2014, at 3:05 PM, Owen DeLong o...@delong.com wrote:
 
 Yes, the more accurate statement would be aggressively seeking new
 ways to monetize the existing infrastructure without investing in upgrades
 or additional buildout any more than absolutely necessary.
 
 Owen
 
 On May 14, 2014, at 8:02 AM, Hugo Slabbert h...@slabnet.com wrote:
 
 
 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin).
 
 Sure they are (seeking new sources of revenue).  They're not necessarily
 creating new products or services, i.e. actually adding any value, but they
 are finding ways to extract additional revenue from the same pipes, e.g.
 through paid peering with content providers.
 
 I'm not endorsing this; just pointing out that you two are actually in
 agreement here.
 
 --
 Hugo
 
 
 On Wed, May 14, 2014 at 7:23 AM, char...@thefnf.org wrote:
 
 On 2014-05-14 02:04, Jean-Francois Mezei wrote:
 
 On 14-05-13 22:50, Daniel Staal wrote:
 
 They have the money.  They have the ability to get more money.  *They see
 no reason to spend money making customers happy.*  They can make more
 profit without it.
 
 There is the issue of control over the market. But also the pressure
 from shareholders for continued growth.
 
 
 Yes. That is true. Except that it's not.
 
 How do service providers grow? Let's explore that:
 
 What is growth for a transit provider?
 
 More (new) access network(s) (connections).
 More bandwidth across backbone pipes.
 
 
 What is growth for access network?
 More subscribers.
 
 Except that the incumbent carriers have shown they have no interest in
 providing decent bandwidth to anywhere but the most profitable rate
 centers. I'd say about 2/3 of the USA is served with quite terrible 
 access.
 
 
 
 
 The problem with the internet is that while it had promises of wild
 growth in the 90s and 00s, once penetration reaches a certain level,
 growth stabilizes.
 
 Penetration is ABYSMAL sir. Huge swaths of underserved americans exist.
 
 
 
 When you combine this with threath to large incumbents's media and media
 distribution endeavours by the likes of Netflix (and cat videos on
 Youtube), large incumbents start thinking about how they will be able to
 continue to grow revenus/profits when customers will shift spending to
 vspecialty channels/cableTV to Netflix and customer growth will not
 compensate.
 
 Except they aren't. Even in the most profitable rate centers, they've
 declined to really invest in the networks. They aren't a real business. 
 You
 have to remember that. They have regulatory capture, natural/defacto
 monopoly etc etc. They don't operate in the real world of
 risk/reward/profit/loss/uncertainty like any other real business has to.
 
 
 
 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin). They 
 did
 a bit of an advertising campaign of smart home offerings, but that seems
 to have never grown beyond a pilot.
 
 
 
 The current trend is to if you can't fight them, jon them where
 cablecos start to include the Netflix app into their proprietary set-top
 boxes. The idea is that you at least make the customer continue to use
 your box and your remote control which makes it easier for them to
 switch between netflix and legacy TV.
 True. I don't know why one of the cablecos hasn't licensed roku, added
 cable card and made that available as a hip/cool set top box offering 
 and
 charge another 10.00 a month on top of the standard dvr rental.
 
 
 
 Would be interesting to see if those cable companies that are agreeing
 to add the Netflix app onto their proprietary STBs also  play peering
 capacity games to degrade the service or not.
 
 So how is the content delivered? Is it over the internet? Or is it over
 the cable

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Scott Berkman

Unfortunately these build-outs are primarily in subscriber facing 
bandwidth and number of headend locations (to add more customers to the 
network).  These peering point/transit connection issues have been going 
on for a long time, evidenced by Level 3 coming out with this post.  
Comcast is also suspiciously absent from public exchanges (TelX's TIE 
would be one example) while many of their competitors participate for 
the benefit of the Internet as a whole and their customers.


Measured broadband is also a game, because its very easy for large 
providers to give priority to (or otherwise help) known speed test and 
similar sites, giving customers a false impression of their available 
capacity or performance.  We've all seen cases where customers have some 
amazing result on their favorite test site, and then real world 
performance can't even come close.


That said, if Comcast does or is making efforts to finally resolve this, 
more power to them and congratulations to their customers. Unfortunately 
trying to brute-force the industry and external content providers tells 
a very different story.  Where is Comcast's official blog post showing 
evidence as to where they do ensure their peering and or transit to the 
largest Tier 1 providers are not congested?  Instead all we see are 
policy arguments about who should pay for what, while users continue to 
suffer.


This is really similar to when TV providers have spats with content 
owners, and the result is the end users missing out on something they 
are paying for.   It is good for related industries and the large 
players in each to keep working with each other in open ways to keep 
pricing reasonable (as opposed to working together in hiding to price 
fix), but it is not OK to do so by throwing tantrums and making everyone 
involved suffer.


  -Scott


On 05/15/2014 10:57 AM, McElearney, Kevin wrote:

Upgrades/buildout are happening every day.  They are continuous to keep ahead 
of demand and publicly measured by SamKnows (FCC measuring broadband), Akamai, 
Ookla, etc

What is not well known is that Comcast has been an existing commercial transit 
business for 15+ years (with over 8000 commercial fiber customers).  Comcast 
also has over 40 balanced peers with plenty of capacity, and some of the 
largest Internet companies as customers.

   - Kevin

215-313-1083


On May 15, 2014, at 10:19 AM, Owen DeLong o...@delong.com wrote:

Oh, please do explicate on how this is inaccurate…

Owen


On May 14, 2014, at 2:14 PM, McElearney, Kevin 
kevin_mcelear...@cable.comcast.com wrote:

Respectfully, this is a highly inaccurate sound bite

- Kevin

215-313-1083


On May 14, 2014, at 3:05 PM, Owen DeLong o...@delong.com wrote:

Yes, the more accurate statement would be aggressively seeking new
ways to monetize the existing infrastructure without investing in upgrades
or additional buildout any more than absolutely necessary.

Owen

On May 14, 2014, at 8:02 AM, Hugo Slabbert h...@slabnet.com wrote:


So they seek new sources of revenues, and/or attempt to thwart

competition any way they can.

No to the first. Yes to the second. If they were seeking new sources of

revenue, they'd be massively expanding into un/der served markets and
aggressively growing over the top services (which are fat margin).

Sure they are (seeking new sources of revenue).  They're not necessarily
creating new products or services, i.e. actually adding any value, but they
are finding ways to extract additional revenue from the same pipes, e.g.
through paid peering with content providers.

I'm not endorsing this; just pointing out that you two are actually in
agreement here.

--
Hugo



On Wed, May 14, 2014 at 7:23 AM, char...@thefnf.org wrote:

On 2014-05-14 02:04, Jean-Francois Mezei wrote:

On 14-05-13 22:50, Daniel Staal wrote:

They have the money.  They have the ability to get more money.  *They see

no reason to spend money making customers happy.*  They can make more
profit without it.

There is the issue of control over the market. But also the pressure
from shareholders for continued growth.


Yes. That is true. Except that it's not.

How do service providers grow? Let's explore that:

What is growth for a transit provider?

More (new) access network(s) (connections).
More bandwidth across backbone pipes.


What is growth for access network?
More subscribers.

Except that the incumbent carriers have shown they have no interest in
providing decent bandwidth to anywhere but the most profitable rate
centers. I'd say about 2/3 of the USA is served with quite terrible access.





The problem with the internet is that while it had promises of wild
growth in the 90s and 00s, once penetration reaches a certain level,
growth stabilizes.

Penetration is ABYSMAL sir. Huge swaths of underserved americans exist.




When you combine this with threath to large incumbents's media and media
distribution endeavours by the likes of Netflix (and cat videos on
Youtube), large

Re: Observations of an Internet Middleman (Level3)

There is no gaming on measurements and disputes are isolated and temporary with 
issues not unique over the history of the internet.  I think all the same 
rhetorical quotes continue to be reused

  - Kevin

 On May 15, 2014, at 11:43 AM, Scott Berkman sc...@sberkman.net wrote:
 
 Unfortunately these build-outs are primarily in subscriber facing bandwidth 
 and number of headend locations (to add more customers to the network).  
 These peering point/transit connection issues have been going on for a long 
 time, evidenced by Level 3 coming out with this post.  Comcast is also 
 suspiciously absent from public exchanges (TelX's TIE would be one example) 
 while many of their competitors participate for the benefit of the Internet 
 as a whole and their customers.
 
 Measured broadband is also a game, because its very easy for large providers 
 to give priority to (or otherwise help) known speed test and similar sites, 
 giving customers a false impression of their available capacity or 
 performance.  We've all seen cases where customers have some amazing result 
 on their favorite test site, and then real world performance can't even come 
 close.
 
 That said, if Comcast does or is making efforts to finally resolve this, more 
 power to them and congratulations to their customers. Unfortunately trying to 
 brute-force the industry and external content providers tells a very 
 different story.  Where is Comcast's official blog post showing evidence as 
 to where they do ensure their peering and or transit to the largest Tier 1 
 providers are not congested?  Instead all we see are policy arguments about 
 who should pay for what, while users continue to suffer.
 
 This is really similar to when TV providers have spats with content owners, 
 and the result is the end users missing out on something they are paying for. 
   It is good for related industries and the large players in each to keep 
 working with each other in open ways to keep pricing reasonable (as opposed 
 to working together in hiding to price fix), but it is not OK to do so by 
 throwing tantrums and making everyone involved suffer.
 
  -Scott
 
 
 On 05/15/2014 10:57 AM, McElearney, Kevin wrote:
 Upgrades/buildout are happening every day.  They are continuous to keep 
 ahead of demand and publicly measured by SamKnows (FCC measuring broadband), 
 Akamai, Ookla, etc
 
 What is not well known is that Comcast has been an existing commercial 
 transit business for 15+ years (with over 8000 commercial fiber customers).  
 Comcast also has over 40 balanced peers with plenty of capacity, and some of 
 the largest Internet companies as customers.
 
   - Kevin
 
 215-313-1083
 
 On May 15, 2014, at 10:19 AM, Owen DeLong o...@delong.com wrote:
 
 Oh, please do explicate on how this is inaccurate…
 
 Owen
 
 On May 14, 2014, at 2:14 PM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.com wrote:
 
 Respectfully, this is a highly inaccurate sound bite
 
- Kevin
 
 215-313-1083
 
 On May 14, 2014, at 3:05 PM, Owen DeLong o...@delong.com wrote:
 
 Yes, the more accurate statement would be aggressively seeking new
 ways to monetize the existing infrastructure without investing in upgrades
 or additional buildout any more than absolutely necessary.
 
 Owen
 
 On May 14, 2014, at 8:02 AM, Hugo Slabbert h...@slabnet.com wrote:
 
 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin).
 Sure they are (seeking new sources of revenue).  They're not necessarily
 creating new products or services, i.e. actually adding any value, but 
 they
 are finding ways to extract additional revenue from the same pipes, e.g.
 through paid peering with content providers.
 
 I'm not endorsing this; just pointing out that you two are actually in
 agreement here.
 
 --
 Hugo
 
 
 On Wed, May 14, 2014 at 7:23 AM, char...@thefnf.org wrote:
 
 On 2014-05-14 02:04, Jean-Francois Mezei wrote:
 
 On 14-05-13 22:50, Daniel Staal wrote:
 
 They have the money.  They have the ability to get more money.  *They 
 see
 no reason to spend money making customers happy.*  They can make more
 profit without it.
 There is the issue of control over the market. But also the pressure
 from shareholders for continued growth.
 
 Yes. That is true. Except that it's not.
 
 How do service providers grow? Let's explore that:
 
 What is growth for a transit provider?
 
 More (new) access network(s) (connections).
 More bandwidth across backbone pipes.
 
 
 What is growth for access network?
 More subscribers.
 
 Except that the incumbent carriers have shown they have no interest in
 providing decent bandwidth to anywhere but the most profitable rate
 centers. I'd say about 2/3 of the USA is served with quite terrible 
 access.
 
 
 
 
 The

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Jared Mauch


On May 15, 2014, at 11:50 AM, McElearney, Kevin 
kevin_mcelear...@cable.comcast.com wrote:

 There is no gaming on measurements and disputes are isolated and temporary 
 with issues not unique over the history of the internet.  I think all the 
 same rhetorical quotes continue to be reused
 

Kevin,

in the past most issues were transient for a few months as both sides got 
complaints, but while at RIPE earlier this week someone commented to me: 
there's no one provider you can buy access from to get a packet-loss free 
connection to all their other business partners/customers.  This hurts the 
entire marketplace when there is persistent congestion.

Some of these issues are related to (as Craig called them) Hypergiants (OTT) 
but others are due to providers having poor capital models so they don't have 
budget for upgrading unless someone pays for that upgrade, vs seeing their 
existing customer base as that source for the capital.

As an engineer, I'm hopeful that those responsible for budgeting will do the 
right thing.  As a greedy capitalist, please pay me more $$$.  It does feel a 
bit like tic-tac-toe with zero players in wargames though, the only way to win 
is to not play [games].

- Jared

Re: Observations of an Internet Middleman (Level3)

This is a smart group. If if that was true I think every internet site / 
service one visits from home would be a negatively impacted.  That is not the 
case

As I said before, Comcast also has over 40 balanced peers with plenty of 
capacity.  Wholesale $$ are very small, highly competitive and only skin in 
the game to promote efficiencies

  - Kevin


 On May 15, 2014, at 12:01 PM, Jared Mauch ja...@puck.nether.net wrote:
 
 
 On May 15, 2014, at 11:50 AM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.com wrote:
 
 There is no gaming on measurements and disputes are isolated and temporary 
 with issues not unique over the history of the internet.  I think all the 
 same rhetorical quotes continue to be reused
 
 Kevin,
 
 in the past most issues were transient for a few months as both sides got 
 complaints, but while at RIPE earlier this week someone commented to me: 
 there's no one provider you can buy access from to get a packet-loss free 
 connection to all their other business partners/customers.  This hurts the 
 entire marketplace when there is persistent congestion.
 
 Some of these issues are related to (as Craig called them) Hypergiants 
 (OTT) but others are due to providers having poor capital models so they 
 don't have budget for upgrading unless someone pays for that upgrade, vs 
 seeing their existing customer base as that source for the capital.
 
 As an engineer, I'm hopeful that those responsible for budgeting will do the 
 right thing.  As a greedy capitalist, please pay me more $$$.  It does feel a 
 bit like tic-tac-toe with zero players in wargames though, the only way to 
 win is to not play [games].
 
 - Jared

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

Yes, you've got some of the largest Internet companies as customers.
Because you told them if you don't pay us, we'll throttle you.  Then you
throttled them.  I'm sorry, not a winning argument.
Nick


On Thu, May 15, 2014 at 10:57 AM, McElearney, Kevin 
kevin_mcelear...@cable.comcast.com wrote:

 Upgrades/buildout are happening every day.  They are continuous to keep
 ahead of demand and publicly measured by SamKnows (FCC measuring
 broadband), Akamai, Ookla, etc

 What is not well known is that Comcast has been an existing commercial
 transit business for 15+ years (with over 8000 commercial fiber customers).
  Comcast also has over 40 balanced peers with plenty of capacity, and some
 of the largest Internet companies as customers.

   - Kevin

 215-313-1083

  On May 15, 2014, at 10:19 AM, Owen DeLong o...@delong.com wrote:
 
  Oh, please do explicate on how this is inaccurate…
 
  Owen
 
  On May 14, 2014, at 2:14 PM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.com wrote:
 
  Respectfully, this is a highly inaccurate sound bite
 
 - Kevin
 
  215-313-1083
 
  On May 14, 2014, at 3:05 PM, Owen DeLong o...@delong.com wrote:
 
  Yes, the more accurate statement would be aggressively seeking new
  ways to monetize the existing infrastructure without investing in
 upgrades
  or additional buildout any more than absolutely necessary.
 
  Owen
 
  On May 14, 2014, at 8:02 AM, Hugo Slabbert h...@slabnet.com wrote:
 
 
  So they seek new sources of revenues, and/or attempt to thwart
  competition any way they can.
  No to the first. Yes to the second. If they were seeking new sources
 of
  revenue, they'd be massively expanding into un/der served markets and
  aggressively growing over the top services (which are fat margin).
 
  Sure they are (seeking new sources of revenue).  They're not
 necessarily
  creating new products or services, i.e. actually adding any value,
 but they
  are finding ways to extract additional revenue from the same pipes,
 e.g.
  through paid peering with content providers.
 
  I'm not endorsing this; just pointing out that you two are actually in
  agreement here.
 
  --
  Hugo
 
 
  On Wed, May 14, 2014 at 7:23 AM, char...@thefnf.org wrote:
 
  On 2014-05-14 02:04, Jean-Francois Mezei wrote:
 
  On 14-05-13 22:50, Daniel Staal wrote:
 
  They have the money.  They have the ability to get more money.
  *They see
  no reason to spend money making customers happy.*  They can make
 more
  profit without it.
 
  There is the issue of control over the market. But also the pressure
  from shareholders for continued growth.
 
 
  Yes. That is true. Except that it's not.
 
  How do service providers grow? Let's explore that:
 
  What is growth for a transit provider?
 
  More (new) access network(s) (connections).
  More bandwidth across backbone pipes.
 
 
  What is growth for access network?
  More subscribers.
 
  Except that the incumbent carriers have shown they have no interest
 in
  providing decent bandwidth to anywhere but the most profitable rate
  centers. I'd say about 2/3 of the USA is served with quite terrible
 access.
 
 
 
 
  The problem with the internet is that while it had promises of wild
  growth in the 90s and 00s, once penetration reaches a certain level,
  growth stabilizes.
 
  Penetration is ABYSMAL sir. Huge swaths of underserved americans
 exist.
 
 
 
  When you combine this with threath to large incumbents's media and
 media
  distribution endeavours by the likes of Netflix (and cat videos on
  Youtube), large incumbents start thinking about how they will be
 able to
  continue to grow revenus/profits when customers will shift spending
 to
  vspecialty channels/cableTV to Netflix and customer growth will not
  compensate.
 
  Except they aren't. Even in the most profitable rate centers, they've
  declined to really invest in the networks. They aren't a real
 business. You
  have to remember that. They have regulatory capture, natural/defacto
  monopoly etc etc. They don't operate in the real world of
  risk/reward/profit/loss/uncertainty like any other real business has
 to.
 
 
 
  So they seek new sources of revenues, and/or attempt to thwart
  competition any way they can.
 
  No to the first. Yes to the second. If they were seeking new sources
 of
  revenue, they'd be massively expanding into un/der served markets and
  aggressively growing over the top services (which are fat margin).
 They did
  a bit of an advertising campaign of smart home offerings, but that
 seems
  to have never grown beyond a pilot.
 
 
 
  The current trend is to if you can't fight them, jon them where
  cablecos start to include the Netflix app into their proprietary
 set-top
  boxes. The idea is that you at least make the customer continue to
 use
  your box and your remote control which makes it easier for them to
  switch between netflix and legacy TV.
  True. I don't know why one of the cablecos hasn't licensed roku,
 added
  cable card and made that available

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

Guys, I'm already pretty far off the reservation and will not respond to 
trolling. I think most ISPs are starting to avoid participation here for the 
same reason. I'm going to stop for a while.

  - Kevin


On May 15, 2014, at 12:42 PM, Nick B 
n...@pelagiris.orgmailto:n...@pelagiris.org wrote:

Yes, you've got some of the largest Internet companies as customers.  Because 
you told them if you don't pay us, we'll throttle you.  Then you throttled 
them.  I'm sorry, not a winning argument.
Nick


On Thu, May 15, 2014 at 10:57 AM, McElearney, Kevin 
kevin_mcelear...@cable.comcast.commailto:kevin_mcelear...@cable.comcast.com 
wrote:
Upgrades/buildout are happening every day.  They are continuous to keep ahead 
of demand and publicly measured by SamKnows (FCC measuring broadband), Akamai, 
Ookla, etc

What is not well known is that Comcast has been an existing commercial transit 
business for 15+ years (with over 8000 commercial fiber customers).  Comcast 
also has over 40 balanced peers with plenty of capacity, and some of the 
largest Internet companies as customers.

  - Kevin

215-313-1083tel:215-313-1083

 On May 15, 2014, at 10:19 AM, Owen DeLong 
 o...@delong.commailto:o...@delong.com wrote:

 Oh, please do explicate on how this is inaccurate…

 Owen

 On May 14, 2014, at 2:14 PM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.commailto:kevin_mcelear...@cable.comcast.com
  wrote:

 Respectfully, this is a highly inaccurate sound bite

- Kevin

 215-313-1083tel:215-313-1083

 On May 14, 2014, at 3:05 PM, Owen DeLong 
 o...@delong.commailto:o...@delong.com wrote:

 Yes, the more accurate statement would be aggressively seeking new
 ways to monetize the existing infrastructure without investing in upgrades
 or additional buildout any more than absolutely necessary.

 Owen

 On May 14, 2014, at 8:02 AM, Hugo Slabbert 
 h...@slabnet.commailto:h...@slabnet.com wrote:


 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin).

 Sure they are (seeking new sources of revenue).  They're not necessarily
 creating new products or services, i.e. actually adding any value, but they
 are finding ways to extract additional revenue from the same pipes, e.g.
 through paid peering with content providers.

 I'm not endorsing this; just pointing out that you two are actually in
 agreement here.

 --
 Hugo


 On Wed, May 14, 2014 at 7:23 AM, 
 char...@thefnf.orgmailto:char...@thefnf.org wrote:

 On 2014-05-14 02:04, Jean-Francois Mezei wrote:

 On 14-05-13 22:50, Daniel Staal wrote:

 They have the money.  They have the ability to get more money.  *They see
 no reason to spend money making customers happy.*  They can make more
 profit without it.

 There is the issue of control over the market. But also the pressure
 from shareholders for continued growth.


 Yes. That is true. Except that it's not.

 How do service providers grow? Let's explore that:

 What is growth for a transit provider?

 More (new) access network(s) (connections).
 More bandwidth across backbone pipes.


 What is growth for access network?
 More subscribers.

 Except that the incumbent carriers have shown they have no interest in
 providing decent bandwidth to anywhere but the most profitable rate
 centers. I'd say about 2/3 of the USA is served with quite terrible 
 access.




 The problem with the internet is that while it had promises of wild
 growth in the 90s and 00s, once penetration reaches a certain level,
 growth stabilizes.

 Penetration is ABYSMAL sir. Huge swaths of underserved americans exist.



 When you combine this with threath to large incumbents's media and media
 distribution endeavours by the likes of Netflix (and cat videos on
 Youtube), large incumbents start thinking about how they will be able to
 continue to grow revenus/profits when customers will shift spending to
 vspecialty channels/cableTV to Netflix and customer growth will not
 compensate.

 Except they aren't. Even in the most profitable rate centers, they've
 declined to really invest in the networks. They aren't a real business. 
 You
 have to remember that. They have regulatory capture, natural/defacto
 monopoly etc etc. They don't operate in the real world of
 risk/reward/profit/loss/uncertainty like any other real business has to.



 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.

 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin). They 
 did
 a bit of an advertising campaign of smart home offerings, but that seems
 to have never grown beyond a pilot.



 The current trend is to if you can't

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Ryan Brooks


On 5/15/14, 11:58 AM, Joe Greco wrote:

2) Netflix purchases 5Mbps fast lane



I appreciate Joe's use of quotation marks here.A lot of the dialog 
has included this 'fast lane' terminology, yet all of us know there's no 
'fast lane' being constructed, rather just varying degrees of _slow_ 
applied to existing traffic.


It's a shame the use of 'fast lane' is ubiquitous in this argument.
If the local distribution networks would like to actually build 
something fast, then this would be a different story.


-Ryan Brooks

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

To be fair, I have no evidence that Comcast demanded money in advance.  As
far as I can tell, Level 3, Cogent and Comcast all agree on the rest
though, Comcast's peering filled up.  Both Level 3 and Cogent
offered/requested to upgrade.  Then at least Cogent (IIRC?) offered to
upgrade *and pay Comcast to upgrade*.  All of these offers were ignored or
rejected, and at this point I think all parties agree that ransom was
demanded by Comcast.  Then, just weeks after Verizon's successful
litigation against Net Neutrality, Netflix agreed to pay Comcast directly.
I'm not sure if the removal of Netflix is enough to resolve the congestion
on the Level 3 and Cogent links, but I'm not sure how one could explain the
above situation other than deliberate throttling in an attempt to extort
money.
Nick


On Thu, May 15, 2014 at 12:51 PM, arvindersi...@mail2tor.com wrote:

 Yes Kevin, this is understood - but valid observation from Nick.

 Can you pls answer my question first?  Very curious.

 Arvinder

  Guys, I'm already pretty far off the reservation and will not respond to
  trolling. I think most ISPs are starting to avoid participation here for
  the same reason. I'm going to stop for a while.
 
- Kevin
 
 
  On May 15, 2014, at 12:42 PM, Nick B
  n...@pelagiris.orgmailto:n...@pelagiris.org wrote:
 
  Yes, you've got some of the largest Internet companies as customers.
  Because you told them if you don't pay us, we'll throttle you.  Then
 you
  throttled them.  I'm sorry, not a winning argument.
  Nick
 
 
  On Thu, May 15, 2014 at 10:57 AM, McElearney, Kevin
  kevin_mcelear...@cable.comcast.commailto:
 kevin_mcelear...@cable.comcast.com
  wrote:
  Upgrades/buildout are happening every day.  They are continuous to keep
  ahead of demand and publicly measured by SamKnows (FCC measuring
  broadband), Akamai, Ookla, etc
 
  What is not well known is that Comcast has been an existing commercial
  transit business for 15+ years (with over 8000 commercial fiber
  customers).  Comcast also has over 40 balanced peers with plenty of
  capacity, and some of the largest Internet companies as customers.
 
- Kevin
 
  215-313-1083tel:215-313-1083
 
  On May 15, 2014, at 10:19 AM, Owen DeLong
  o...@delong.commailto:o...@delong.com wrote:
 
  Oh, please do explicate on how this is inaccurate…
 
  Owen
 
  On May 14, 2014, at 2:14 PM, McElearney, Kevin
  kevin_mcelear...@cable.comcast.commailto:
 kevin_mcelear...@cable.comcast.com
  wrote:
 
  Respectfully, this is a highly inaccurate sound bite
 
 - Kevin
 
  215-313-1083tel:215-313-1083
 
  On May 14, 2014, at 3:05 PM, Owen DeLong
  o...@delong.commailto:o...@delong.com wrote:
 
  Yes, the more accurate statement would be aggressively seeking new
  ways to monetize the existing infrastructure without investing in
  upgrades
  or additional buildout any more than absolutely necessary.
 
  Owen
 
  On May 14, 2014, at 8:02 AM, Hugo Slabbert
  h...@slabnet.commailto:h...@slabnet.com wrote:
 
 
  So they seek new sources of revenues, and/or attempt to thwart
  competition any way they can.
  No to the first. Yes to the second. If they were seeking new sources
  of
  revenue, they'd be massively expanding into un/der served markets
  and
  aggressively growing over the top services (which are fat margin).
 
  Sure they are (seeking new sources of revenue).  They're not
  necessarily
  creating new products or services, i.e. actually adding any value,
  but they
  are finding ways to extract additional revenue from the same pipes,
  e.g.
  through paid peering with content providers.
 
  I'm not endorsing this; just pointing out that you two are actually
  in
  agreement here.
 
  --
  Hugo
 
 
  On Wed, May 14, 2014 at 7:23 AM,
  char...@thefnf.orgmailto:char...@thefnf.org wrote:
 
  On 2014-05-14 02:04, Jean-Francois Mezei wrote:
 
  On 14-05-13 22:50, Daniel Staal wrote:
 
  They have the money.  They have the ability to get more money.
  *They see
  no reason to spend money making customers happy.*  They can make
  more
  profit without it.
 
  There is the issue of control over the market. But also the
  pressure
  from shareholders for continued growth.
 
 
  Yes. That is true. Except that it's not.
 
  How do service providers grow? Let's explore that:
 
  What is growth for a transit provider?
 
  More (new) access network(s) (connections).
  More bandwidth across backbone pipes.
 
 
  What is growth for access network?
  More subscribers.
 
  Except that the incumbent carriers have shown they have no interest
  in
  providing decent bandwidth to anywhere but the most profitable rate
  centers. I'd say about 2/3 of the USA is served with quite terrible
  access.
 
 
 
 
  The problem with the internet is that while it had promises of wild
  growth in the 90s and 00s, once penetration reaches a certain
  level,
  growth stabilizes.
 
  Penetration is ABYSMAL sir. Huge swaths of underserved americans
  exist.
 
 
 
  When you combine this with threath to

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

2014-05-15 Thread Jerry Dent

 What is not well known is that Comcast has been an existing commercial
transit business for 15+ years (with over 8000 commercial fiber customers).
 Comcast also has over 40 balanced peers with plenty of capacity, and some
of the largest Internet companies as customers.


Peers that are balanced only due strong arm tactics used against smaller
hosting and content providers.


On Thu, May 15, 2014 at 11:46 AM, McElearney, Kevin 
kevin_mcelear...@cable.comcast.com wrote:

 Guys, I'm already pretty far off the reservation and will not respond to
 trolling. I think most ISPs are starting to avoid participation here for
 the same reason. I'm going to stop for a while.

   - Kevin


 On May 15, 2014, at 12:42 PM, Nick B n...@pelagiris.orgmailto:
 n...@pelagiris.org wrote:

 Yes, you've got some of the largest Internet companies as customers.
  Because you told them if you don't pay us, we'll throttle you.  Then you
 throttled them.  I'm sorry, not a winning argument.
 Nick


 On Thu, May 15, 2014 at 10:57 AM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.commailto:
 kevin_mcelear...@cable.comcast.com wrote:
 Upgrades/buildout are happening every day.  They are continuous to keep
 ahead of demand and publicly measured by SamKnows (FCC measuring
 broadband), Akamai, Ookla, etc

 What is not well known is that Comcast has been an existing commercial
 transit business for 15+ years (with over 8000 commercial fiber customers).
  Comcast also has over 40 balanced peers with plenty of capacity, and some
 of the largest Internet companies as customers.

   - Kevin

 215-313-1083tel:215-313-1083

  On May 15, 2014, at 10:19 AM, Owen DeLong o...@delong.commailto:
 o...@delong.com wrote:
 
  Oh, please do explicate on how this is inaccurate…
 
  Owen
 
  On May 14, 2014, at 2:14 PM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.commailto:
 kevin_mcelear...@cable.comcast.com wrote:
 
  Respectfully, this is a highly inaccurate sound bite
 
 - Kevin
 
  215-313-1083tel:215-313-1083
 
  On May 14, 2014, at 3:05 PM, Owen DeLong o...@delong.commailto:
 o...@delong.com wrote:
 
  Yes, the more accurate statement would be aggressively seeking new
  ways to monetize the existing infrastructure without investing in
 upgrades
  or additional buildout any more than absolutely necessary.
 
  Owen
 
  On May 14, 2014, at 8:02 AM, Hugo Slabbert h...@slabnet.commailto:
 h...@slabnet.com wrote:
 
 
  So they seek new sources of revenues, and/or attempt to thwart
  competition any way they can.
  No to the first. Yes to the second. If they were seeking new sources
 of
  revenue, they'd be massively expanding into un/der served markets and
  aggressively growing over the top services (which are fat margin).
 
  Sure they are (seeking new sources of revenue).  They're not
 necessarily
  creating new products or services, i.e. actually adding any value,
 but they
  are finding ways to extract additional revenue from the same pipes,
 e.g.
  through paid peering with content providers.
 
  I'm not endorsing this; just pointing out that you two are actually in
  agreement here.
 
  --
  Hugo
 
 
  On Wed, May 14, 2014 at 7:23 AM, char...@thefnf.orgmailto:
 char...@thefnf.org wrote:
 
  On 2014-05-14 02:04, Jean-Francois Mezei wrote:
 
  On 14-05-13 22:50, Daniel Staal wrote:
 
  They have the money.  They have the ability to get more money.
  *They see
  no reason to spend money making customers happy.*  They can make
 more
  profit without it.
 
  There is the issue of control over the market. But also the pressure
  from shareholders for continued growth.
 
 
  Yes. That is true. Except that it's not.
 
  How do service providers grow? Let's explore that:
 
  What is growth for a transit provider?
 
  More (new) access network(s) (connections).
  More bandwidth across backbone pipes.
 
 
  What is growth for access network?
  More subscribers.
 
  Except that the incumbent carriers have shown they have no interest
 in
  providing decent bandwidth to anywhere but the most profitable rate
  centers. I'd say about 2/3 of the USA is served with quite terrible
 access.
 
 
 
 
  The problem with the internet is that while it had promises of wild
  growth in the 90s and 00s, once penetration reaches a certain level,
  growth stabilizes.
 
  Penetration is ABYSMAL sir. Huge swaths of underserved americans
 exist.
 
 
 
  When you combine this with threath to large incumbents's media and
 media
  distribution endeavours by the likes of Netflix (and cat videos on
  Youtube), large incumbents start thinking about how they will be
 able to
  continue to grow revenus/profits when customers will shift spending
 to
  vspecialty channels/cableTV to Netflix and customer growth will not
  compensate.
 
  Except they aren't. Even in the most profitable rate centers, they've
  declined to really invest in the networks. They aren't a real
 business. You
  have to remember that. They have regulatory capture, natural/defacto
  monopoly etc

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Blake Dunlap

And the unbalanced peers / transit?

-Blake

On Thu, May 15, 2014 at 11:41 AM, McElearney, Kevin
kevin_mcelear...@cable.comcast.com wrote:
 This is a smart group. If if that was true I think every internet site / 
 service one visits from home would be a negatively impacted.  That is not the 
 case

 As I said before, Comcast also has over 40 balanced peers with plenty of 
 capacity.  Wholesale $$ are very small, highly competitive and only skin in 
 the game to promote efficiencies

   - Kevin


 On May 15, 2014, at 12:01 PM, Jared Mauch ja...@puck.nether.net wrote:


 On May 15, 2014, at 11:50 AM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.com wrote:

 There is no gaming on measurements and disputes are isolated and temporary 
 with issues not unique over the history of the internet.  I think all the 
 same rhetorical quotes continue to be reused

 Kevin,

 in the past most issues were transient for a few months as both sides got 
 complaints, but while at RIPE earlier this week someone commented to me: 
 there's no one provider you can buy access from to get a packet-loss free 
 connection to all their other business partners/customers.  This hurts the 
 entire marketplace when there is persistent congestion.

 Some of these issues are related to (as Craig called them) Hypergiants 
 (OTT) but others are due to providers having poor capital models so they 
 don't have budget for upgrading unless someone pays for that upgrade, vs 
 seeing their existing customer base as that source for the capital.

 As an engineer, I'm hopeful that those responsible for budgeting will do the 
 right thing.  As a greedy capitalist, please pay me more $$$.  It does feel 
 a bit like tic-tac-toe with zero players in wargames though, the only way to 
 win is to not play [games].

 - Jared

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

2014-05-15 Thread Jean-Francois Mezei

On 14-05-15 10:26, Owen DeLong wrote:
 Choosing between Comcast and a legacy Telco is like choosing between 
 legionnaire’s disease and SARS.

Twisted pair is certantly legacy.

Is there a feeling that coax cable/DOSCIS is also legacy in terms of
current capacity/speeds ? Or is that technology still considered viable
against FTTH ?

I realise that business practices make north american incumbents
undesirable compared to the rest of the world, especially Comcast's
dirty tricks with Netflix as an example.

But in terms of the last mile technology and wiring (for instance, homes
per HFC node) sre north american cavlecos up to par with the rest of the
world ?

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality


On May 15, 2014, at 7:57 AM, McElearney, Kevin 
kevin_mcelear...@cable.comcast.com wrote:

 Upgrades/buildout are happening every day.  They are continuous to keep ahead 
 of demand and publicly measured by SamKnows (FCC measuring broadband), 
 Akamai, Ookla, etc

I didn’t say they weren’t doing any upgrades/buildouts.

I will say that the copper capabilities in my neighborhood are so far behind 
demand(s) that it is abysmal. There hasn’t been significant maintenance to the 
$TELCO copper plant in my neighborhood since it was installed in 1960.

 What is not well known is that Comcast has been an existing commercial 
 transit business for 15+ years (with over 8000 commercial fiber customers).  
 Comcast also has over 40 balanced peers with plenty of capacity, and some of 
 the largest Internet companies as customers.

I’ve been asked by my employer to stop picking on specific large ISPs. However, 
my experiences with $CABLECO have been as described. The infrastructure in my 
neighborhood was horrible and did not improve at all until I ordered business 
class service from them. I’ve seen nothing to indicate that there is any 
significant effort to improve customer satisfaction, but lots of things to 
indicate that they are trying to leverage as much revenue out of as little 
investment as possible.

Owen

 
  - Kevin
 
 215-313-1083
 
 On May 15, 2014, at 10:19 AM, Owen DeLong o...@delong.com wrote:
 
 Oh, please do explicate on how this is inaccurate…
 
 Owen
 
 On May 14, 2014, at 2:14 PM, McElearney, Kevin 
 kevin_mcelear...@cable.comcast.com wrote:
 
 Respectfully, this is a highly inaccurate sound bite
 
   - Kevin
 
 215-313-1083
 
 On May 14, 2014, at 3:05 PM, Owen DeLong o...@delong.com wrote:
 
 Yes, the more accurate statement would be aggressively seeking new
 ways to monetize the existing infrastructure without investing in upgrades
 or additional buildout any more than absolutely necessary.
 
 Owen
 
 On May 14, 2014, at 8:02 AM, Hugo Slabbert h...@slabnet.com wrote:
 
 
 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin).
 
 Sure they are (seeking new sources of revenue).  They're not necessarily
 creating new products or services, i.e. actually adding any value, but 
 they
 are finding ways to extract additional revenue from the same pipes, e.g.
 through paid peering with content providers.
 
 I'm not endorsing this; just pointing out that you two are actually in
 agreement here.
 
 --
 Hugo
 
 
 On Wed, May 14, 2014 at 7:23 AM, char...@thefnf.org wrote:
 
 On 2014-05-14 02:04, Jean-Francois Mezei wrote:
 
 On 14-05-13 22:50, Daniel Staal wrote:
 
 They have the money.  They have the ability to get more money.  *They 
 see
 no reason to spend money making customers happy.*  They can make more
 profit without it.
 
 There is the issue of control over the market. But also the pressure
 from shareholders for continued growth.
 
 
 Yes. That is true. Except that it's not.
 
 How do service providers grow? Let's explore that:
 
 What is growth for a transit provider?
 
 More (new) access network(s) (connections).
 More bandwidth across backbone pipes.
 
 
 What is growth for access network?
 More subscribers.
 
 Except that the incumbent carriers have shown they have no interest in
 providing decent bandwidth to anywhere but the most profitable rate
 centers. I'd say about 2/3 of the USA is served with quite terrible 
 access.
 
 
 
 
 The problem with the internet is that while it had promises of wild
 growth in the 90s and 00s, once penetration reaches a certain level,
 growth stabilizes.
 
 Penetration is ABYSMAL sir. Huge swaths of underserved americans exist.
 
 
 
 When you combine this with threath to large incumbents's media and media
 distribution endeavours by the likes of Netflix (and cat videos on
 Youtube), large incumbents start thinking about how they will be able to
 continue to grow revenus/profits when customers will shift spending to
 vspecialty channels/cableTV to Netflix and customer growth will not
 compensate.
 
 Except they aren't. Even in the most profitable rate centers, they've
 declined to really invest in the networks. They aren't a real business. 
 You
 have to remember that. They have regulatory capture, natural/defacto
 monopoly etc etc. They don't operate in the real world of
 risk/reward/profit/loss/uncertainty like any other real business has to.
 
 
 
 So they seek new sources of revenues, and/or attempt to thwart
 competition any way they can.
 
 No to the first. Yes to the second. If they were seeking new sources of
 revenue, they'd be massively expanding into un/der served markets and
 aggressively growing over the top services (which are fat margin). They 
 did
 a bit of an advertising campaign of smart

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Paul Ferguson

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 5/15/2014 10:06 AM, Ryan Brooks wrote:

 It's a shame the use of 'fast lane' is ubiquitous in this argument.
  If the local distribution networks would like to actually build 
 something fast, then this would be a different story.

Okay, then call it the faster lane or the uncongested lane or
something that actually reflects bias and preferential treatment. It's
a done deal now:

http://www.washingtonpost.com/blogs/the-switch/wp/2014/05/15/fcc-approves-plan-to-allow-for-paid-priority-on-internet

FYI,

- - ferg


- -- 
Paul Ferguson
VP Threat Intelligence, IID
PGP Public Key ID: 0x54DC85B2
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlN09roACgkQKJasdVTchbJPlgEAtBpp0TKNZcIdrDkVP75Tni7a
O9PvcnKZdaPNuNUpOb0A/RQ5hvrqPAu/QLSp8dPbcDSO5Zad8Z3JG67UfI6yaeJH
=DhnX
-END PGP SIGNATURE-

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Christopher Morrow

On Thu, May 15, 2014 at 1:06 PM, Ryan Brooks r...@hack.net wrote:
 On 5/15/14, 11:58 AM, Joe Greco wrote:

 2) Netflix purchases 5Mbps fast lane


 I appreciate Joe's use of quotation marks here.A lot of the dialog has
 included this 'fast lane' terminology, yet all of us know there's no 'fast
 lane' being constructed, rather just varying degrees of _slow_ applied to
 existing traffic.


please correct me if I'm wrong, but 'fast lane' really is (in this example):
  'cableco' port from 'moviecompany' has 'qos' marking configuration
to set all 'moviecompany' traffic (from this port!) to some priority
level.

  customer-port to 'cableco' has 'qos' handling/queuing that will
ensure '5mbps' of 'moviecompany' is always going to get down the link
to the customer, regardless of the other traffic the customer is
requesting.

right? (presume that in the rest of the 'cableco' network is
protecting 'moviecompany' traffic as well, of course)

So, when there are 1 'moviecompany' things to prioritize and deliver
that's cool... but what about when there are 10? 100? 1000? doesn't
the queuing get complicated? what if the 'cableco' customer with
10mbps link has 3 people in the location all streaming from 3
different 'moviecompany' organizations which have paid for 'fastlane'
services?

3 x 5 == 15 ... not 10. How will 'cableco' manage this when their
100gbps inter-metro links are seeing +100gbps if 'fastlane' traffic
and 'fastlane' traffic can't make it to the local metro from the
remote one?

This all seems much, much more complicated and expensive than just
building out networking, which they will have to do in the end anyway,
right? Only with 'fastlanes' there's extra capacity management and
configuration and testing and ... all on top of: Gosh, does the new
umnptyfart card from routerco actually work in old routerco routers?

This looks, to me, like nuttiness... like mutually assured destruction
that the cableco folk are driving both parties into intentionally.

-chris

BTW: I didn't use a particular 'cable company' name for 'cableco', nor
did I use a particular streaming media company for 'moviecompany'...
Also, 'cableco' is short-hand for
'lastmile-consumer-provider-network'. Less typing was better, for me,
I thought.

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

On 5/15/14, 12:43 PM, Nick B n...@pelagiris.org wrote:


Yes, you've got some of the largest Internet companies as customers².
Because you told them if you don't pay us, we'll throttle you.  Then
you throttled them.  I'm sorry, not a winning argument.
Nick

That is categorically untrue, however nice a soundbite it may be. If you
or anyone else truly believes we are throttling someone then I encourage
you to file a formal complaint with the FCC. According to their Open
Internet rules that we are bound to through at least 2018 (IIRC) we may
not discriminate on traffic in that way, so there is a clear rule and a
clear process for complaints.

Jason

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

By categorically untrue do you mean FCC's open internet rules allow us
to refuse to upgrade full peers?
Nick


On Thu, May 15, 2014 at 1:26 PM, Livingood, Jason 
jason_living...@cable.comcast.com wrote:

 On 5/15/14, 12:43 PM, Nick B n...@pelagiris.org wrote:


 Yes, you've got some of the largest Internet companies as customers².
 Because you told them if you don't pay us, we'll throttle you.  Then
 you throttled them.  I'm sorry, not a winning argument.
 Nick

 That is categorically untrue, however nice a soundbite it may be. If you
 or anyone else truly believes we are throttling someone then I encourage
 you to file a formal complaint with the FCC. According to their Open
 Internet rules that we are bound to through at least 2018 (IIRC) we may
 not discriminate on traffic in that way, so there is a clear rule and a
 clear process for complaints.

 Jason

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

On 5/15/14, 1:28 PM, Nick B n...@pelagiris.orgmailto:n...@pelagiris.org 
wrote:

By categorically untrue do you mean FCC's open internet rules allow us to 
refuse to upgrade full peers?

Throttling is taking, say, a link from 10G and applying policy to constrain it 
to 1G, for example. What if a peer wants to go from a balanced relationship to 
10,000:1, well outside of the policy binding the relationship? Should we just 
unquestionably toss out our published policy – which is consistent with other 
networks – and ignore expectations for other peers?

Jason

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

Yes, throttling an entire ISP by refusing to upgrade peering is clearly a
way to avoid technically throttling.  Interestingly enough only Comcast and
Verizon are having this problem, though I'm sure now that you have set an
example others will follow.
Nick


On Thu, May 15, 2014 at 1:34 PM, Livingood, Jason 
jason_living...@cable.comcast.com wrote:

   On 5/15/14, 1:28 PM, Nick B n...@pelagiris.org wrote:

   By categorically untrue do you mean FCC's open internet rules allow
 us to refuse to upgrade full peers?


  Throttling is taking, say, a link from 10G and applying policy to
 constrain it to 1G, for example. What if a peer wants to go from a balanced
 relationship to 10,000:1, well outside of the policy binding the
 relationship? Should we just unquestionably toss out our published policy –
 which is consistent with other networks – and ignore expectations for other
 peers?

  Jason

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Scott Helms

Its not really that complex, if you think about it having 1s of
'movieco' with the same priority is the status quo.  At the end of the day
the QoS mechanics in DOCSIS are pretty straightforward and rely on service
flows, while service flows can have equal priority I doubt most operators
will sell more than a few (perhaps just one) top priority in a given a
category.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Thu, May 15, 2014 at 1:22 PM, Christopher Morrow morrowc.li...@gmail.com
 wrote:

 On Thu, May 15, 2014 at 1:06 PM, Ryan Brooks r...@hack.net wrote:
  On 5/15/14, 11:58 AM, Joe Greco wrote:
 
  2) Netflix purchases 5Mbps fast lane
 
 
  I appreciate Joe's use of quotation marks here.A lot of the dialog
 has
  included this 'fast lane' terminology, yet all of us know there's no
 'fast
  lane' being constructed, rather just varying degrees of _slow_ applied to
  existing traffic.
 

 please correct me if I'm wrong, but 'fast lane' really is (in this
 example):
   'cableco' port from 'moviecompany' has 'qos' marking configuration
 to set all 'moviecompany' traffic (from this port!) to some priority
 level.

   customer-port to 'cableco' has 'qos' handling/queuing that will
 ensure '5mbps' of 'moviecompany' is always going to get down the link
 to the customer, regardless of the other traffic the customer is
 requesting.

 right? (presume that in the rest of the 'cableco' network is
 protecting 'moviecompany' traffic as well, of course)

 So, when there are 1 'moviecompany' things to prioritize and deliver
 that's cool... but what about when there are 10? 100? 1000? doesn't
 the queuing get complicated? what if the 'cableco' customer with
 10mbps link has 3 people in the location all streaming from 3
 different 'moviecompany' organizations which have paid for 'fastlane'
 services?

 3 x 5 == 15 ... not 10. How will 'cableco' manage this when their
 100gbps inter-metro links are seeing +100gbps if 'fastlane' traffic
 and 'fastlane' traffic can't make it to the local metro from the
 remote one?

 This all seems much, much more complicated and expensive than just
 building out networking, which they will have to do in the end anyway,
 right? Only with 'fastlanes' there's extra capacity management and
 configuration and testing and ... all on top of: Gosh, does the new
 umnptyfart card from routerco actually work in old routerco routers?

 This looks, to me, like nuttiness... like mutually assured destruction
 that the cableco folk are driving both parties into intentionally.

 -chris

 BTW: I didn't use a particular 'cable company' name for 'cableco', nor
 did I use a particular streaming media company for 'moviecompany'...
 Also, 'cableco' is short-hand for
 'lastmile-consumer-provider-network'. Less typing was better, for me,
 I thought.

Re: Observations of an Internet Middleman (Level3)

Blake Dunlap iki...@gmail.com wrote:
 And the unbalanced peers / transit?

Surely it is too much to expect a service provider to actually provide
service even if it is not entirely fair and balanced.  It's not like, 
you know, anyone was paying them to provide a service ...


[...rewind...]

kevin_mcelear...@cable.comcast.com wrote:
 This is a smart group.

Well, smart enough to at least try to see it for what it actually is.
Telling us we're smart and then expecting us to swallow a load doesn't
quite seem to work, judging from the last few responses you've had.


Some of us are actually businesspeople, so we understand the issues 
from multiple dimensions.  Including historical ones, where there are
both examples of Monopolies Gone Wild! (Spring Break Edition!) and 
also Government Regulation Gone Overboard.  If you'd like to say that
you're trying to leverage as much revenue as possible by taking advantage
of new trends (i.e. cord cutting) in a customer base that's at least 
partially without other reasonable options, while keeping investment 
costs as low as possible, well, then, we have the potential for an
honest conversation.

But if you're going to tell us about how you've managed to acquire
transit customers, that feels like the start of a dishonest discussion
because basically most of us here wouldn't buy transit from a cable
company unless it was the only available option, or there was some
other distorting reason - such as congestion - that caused such an
arrangement to be needed.


... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Christopher Morrow

On Thu, May 15, 2014 at 1:48 PM, Scott Helms khe...@zcorum.com wrote:
 Its not really that complex, if you think about it having 1s of
 'movieco' with the same priority is the status quo.  At the end of the day
 the QoS mechanics in DOCSIS are pretty straightforward and rely on service
 flows, while service flows can have equal priority I doubt most operators
 will sell more than a few (perhaps just one) top priority in a given a
 category.


yes, there will only ever be 5 computers. or you couldn't possibly
need more than 640kb of ram. or more than 4billion 'ip addresses'.

I don't think you have to get to more than 10 or 20 of the stated
examples before things get dicey ... Once a set of customers
experience (and can measure) the effect, they'll back their complaints
up to 'moviecompany' and some set of contract penalties will kick in,
I suspect.

Sure, if there is only one it's not a problem, but there are already
not just one...


 Scott Helms
 Vice President of Technology
 ZCorum
 (678) 507-5000
 
 http://twitter.com/kscotthelms
 


 On Thu, May 15, 2014 at 1:22 PM, Christopher Morrow
 morrowc.li...@gmail.com wrote:

 On Thu, May 15, 2014 at 1:06 PM, Ryan Brooks r...@hack.net wrote:
  On 5/15/14, 11:58 AM, Joe Greco wrote:
 
  2) Netflix purchases 5Mbps fast lane
 
 
  I appreciate Joe's use of quotation marks here.A lot of the dialog
  has
  included this 'fast lane' terminology, yet all of us know there's no
  'fast
  lane' being constructed, rather just varying degrees of _slow_ applied
  to
  existing traffic.
 

 please correct me if I'm wrong, but 'fast lane' really is (in this
 example):
   'cableco' port from 'moviecompany' has 'qos' marking configuration
 to set all 'moviecompany' traffic (from this port!) to some priority
 level.

   customer-port to 'cableco' has 'qos' handling/queuing that will
 ensure '5mbps' of 'moviecompany' is always going to get down the link
 to the customer, regardless of the other traffic the customer is
 requesting.

 right? (presume that in the rest of the 'cableco' network is
 protecting 'moviecompany' traffic as well, of course)

 So, when there are 1 'moviecompany' things to prioritize and deliver
 that's cool... but what about when there are 10? 100? 1000? doesn't
 the queuing get complicated? what if the 'cableco' customer with
 10mbps link has 3 people in the location all streaming from 3
 different 'moviecompany' organizations which have paid for 'fastlane'
 services?

 3 x 5 == 15 ... not 10. How will 'cableco' manage this when their
 100gbps inter-metro links are seeing +100gbps if 'fastlane' traffic
 and 'fastlane' traffic can't make it to the local metro from the
 remote one?

 This all seems much, much more complicated and expensive than just
 building out networking, which they will have to do in the end anyway,
 right? Only with 'fastlanes' there's extra capacity management and
 configuration and testing and ... all on top of: Gosh, does the new
 umnptyfart card from routerco actually work in old routerco routers?

 This looks, to me, like nuttiness... like mutually assured destruction
 that the cableco folk are driving both parties into intentionally.

 -chris

 BTW: I didn't use a particular 'cable company' name for 'cableco', nor
 did I use a particular streaming media company for 'moviecompany'...
 Also, 'cableco' is short-hand for
 'lastmile-consumer-provider-network'. Less typing was better, for me,
 I thought.

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Scott Helms

Chris,

You're not reading what I said, nor did I make a statement anything like
 one of the silly things you referenced (640k ram etc).  Prioritization
isn't that complex and today we handle the maximum amount of complexity
already since everything is the same priority right now.

You're trying to make the statement that giving multiple content providers
priority somehow makes connectivity unworkable for consumers as if we don't
have this problem already.  Consumers can easily starve themselves of
bandwidth with video or any other content and almost no connections in the
US have any sort of intelligent fair usage buffering provided by the
service provider.  This is true for both cable, telco, and other operators.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Thu, May 15, 2014 at 2:01 PM, Christopher Morrow morrowc.li...@gmail.com
 wrote:

 On Thu, May 15, 2014 at 1:48 PM, Scott Helms khe...@zcorum.com wrote:
  Its not really that complex, if you think about it having 1s of
  'movieco' with the same priority is the status quo.  At the end of the
 day
  the QoS mechanics in DOCSIS are pretty straightforward and rely on
 service
  flows, while service flows can have equal priority I doubt most operators
  will sell more than a few (perhaps just one) top priority in a given a
  category.
 

 yes, there will only ever be 5 computers. or you couldn't possibly
 need more than 640kb of ram. or more than 4billion 'ip addresses'.

 I don't think you have to get to more than 10 or 20 of the stated
 examples before things get dicey ... Once a set of customers
 experience (and can measure) the effect, they'll back their complaints
 up to 'moviecompany' and some set of contract penalties will kick in,
 I suspect.

 Sure, if there is only one it's not a problem, but there are already
 not just one...

 
  Scott Helms
  Vice President of Technology
  ZCorum
  (678) 507-5000
  
  http://twitter.com/kscotthelms
  
 
 
  On Thu, May 15, 2014 at 1:22 PM, Christopher Morrow
  morrowc.li...@gmail.com wrote:
 
  On Thu, May 15, 2014 at 1:06 PM, Ryan Brooks r...@hack.net wrote:
   On 5/15/14, 11:58 AM, Joe Greco wrote:
  
   2) Netflix purchases 5Mbps fast lane
  
  
   I appreciate Joe's use of quotation marks here.A lot of the dialog
   has
   included this 'fast lane' terminology, yet all of us know there's no
   'fast
   lane' being constructed, rather just varying degrees of _slow_ applied
   to
   existing traffic.
  
 
  please correct me if I'm wrong, but 'fast lane' really is (in this
  example):
'cableco' port from 'moviecompany' has 'qos' marking configuration
  to set all 'moviecompany' traffic (from this port!) to some priority
  level.
 
customer-port to 'cableco' has 'qos' handling/queuing that will
  ensure '5mbps' of 'moviecompany' is always going to get down the link
  to the customer, regardless of the other traffic the customer is
  requesting.
 
  right? (presume that in the rest of the 'cableco' network is
  protecting 'moviecompany' traffic as well, of course)
 
  So, when there are 1 'moviecompany' things to prioritize and deliver
  that's cool... but what about when there are 10? 100? 1000? doesn't
  the queuing get complicated? what if the 'cableco' customer with
  10mbps link has 3 people in the location all streaming from 3
  different 'moviecompany' organizations which have paid for 'fastlane'
  services?
 
  3 x 5 == 15 ... not 10. How will 'cableco' manage this when their
  100gbps inter-metro links are seeing +100gbps if 'fastlane' traffic
  and 'fastlane' traffic can't make it to the local metro from the
  remote one?
 
  This all seems much, much more complicated and expensive than just
  building out networking, which they will have to do in the end anyway,
  right? Only with 'fastlanes' there's extra capacity management and
  configuration and testing and ... all on top of: Gosh, does the new
  umnptyfart card from routerco actually work in old routerco routers?
 
  This looks, to me, like nuttiness... like mutually assured destruction
  that the cableco folk are driving both parties into intentionally.
 
  -chris
 
  BTW: I didn't use a particular 'cable company' name for 'cableco', nor
  did I use a particular streaming media company for 'moviecompany'...
  Also, 'cableco' is short-hand for
  'lastmile-consumer-provider-network'. Less typing was better, for me,
  I thought.

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

So by extension, if you enter an agreement and promise to remain balanced you 
can just willfully throw that out and abuse the heck out of it? Where does it 
end? Why even bother having peering policies at all then?

To use an analogy, if you and I agree to buy a car together and agree to switch 
off who uses it every other day, can I just say forget our agreement – I’m 
just going to drive the car myself every single day – its all mine”?

And as you say, “interestingly enough only Comcast and Verizon are having this 
problem” someone else might say “interestingly enough one content distributor 
is at the center of all of these issues.” I’m frankly surprised that no one is 
stepping back to try to understand what was and is driving those changes.

Jason

On 5/15/14, 1:43 PM, Nick B n...@pelagiris.orgmailto:n...@pelagiris.org 
wrote:

Yes, throttling an entire ISP by refusing to upgrade peering is clearly a way 
to avoid technically throttling.  Interestingly enough only Comcast and Verizon 
are having this problem, though I'm sure now that you have set an example 
others will follow.
Nick

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

I said I would step away, but trying to keep some level of emotion out of 
this...  We all need rational actor behavior in the ecosystem. We need our 
policies and agree to live up to those policies between players. Random and 
inconsistent behavior does not build a well functioning market and is the root 
of most disputes

We can argue about what the policy should be, the impacts, etc and that is a 
fair discussion.

  - Kevin

215-313-1083

On May 15, 2014, at 2:11 PM, Livingood, Jason 
jason_living...@cable.comcast.commailto:jason_living...@cable.comcast.com 
wrote:

So by extension, if you enter an agreement and promise to remain balanced you 
can just willfully throw that out and abuse the heck out of it? Where does it 
end? Why even bother having peering policies at all then?

To use an analogy, if you and I agree to buy a car together and agree to switch 
off who uses it every other day, can I just say forget our agreement – I’m 
just going to drive the car myself every single day – its all mine”?

And as you say, “interestingly enough only Comcast and Verizon are having this 
problem” someone else might say “interestingly enough one content distributor 
is at the center of all of these issues.” I’m frankly surprised that no one is 
stepping back to try to understand what was and is driving those changes.

Jason

On 5/15/14, 1:43 PM, Nick B n...@pelagiris.orgmailto:n...@pelagiris.org 
wrote:

Yes, throttling an entire ISP by refusing to upgrade peering is clearly a way 
to avoid technically throttling.  Interestingly enough only Comcast and Verizon 
are having this problem, though I'm sure now that you have set an example 
others will follow.
Nick

Re: Observations of an Internet Middleman (Level3)

 On Thu, May 15, 2014 at 1:06 PM, Ryan Brooks r...@hack.net wrote:
  On 5/15/14, 11:58 AM, Joe Greco wrote:
  2) Netflix purchases 5Mbps fast lane
 
  I appreciate Joe's use of quotation marks here.A lot of the dialog has
  included this 'fast lane' terminology, yet all of us know there's no 'fast
  lane' being constructed, rather just varying degrees of _slow_ applied to
  existing traffic.
 
 please correct me if I'm wrong, but 'fast lane' really is (in this example):
   'cableco' port from 'moviecompany' has 'qos' marking configuration
 to set all 'moviecompany' traffic (from this port!) to some priority
 level.

I think that's a possibility, but that we're actually talking at a
less-technical level.

[...]

 3 x 5 == 15 ... not 10. How will 'cableco' manage this when their
 100gbps inter-metro links are seeing +100gbps if 'fastlane' traffic
 and 'fastlane' traffic can't make it to the local metro from the
 remote one?

#whocares

You've made a technical implementation issue out of a mostly non-tech
issue.

 This all seems much, much more complicated and expensive than just
 building out networking, which they will have to do in the end anyway,
 right? Only with 'fastlanes' there's extra capacity management and
 configuration and testing and ... all on top of: Gosh, does the new
 umnptyfart card from routerco actually work in old routerco routers?

I certainly agree.  This isn't a technical issue though.  A majority
of the people on this list should appreciate the costs associated with
building and maintaining networks, and there are lots of them to be
sure.  This is about other aspects of the business.

 This looks, to me, like nuttiness... like mutually assured destruction
 that the cableco folk are driving both parties into intentionally.

No.  I don't actually believe that.

Businesses are in the habit of making money.  There's a reasonably 
strong desire to remain in business and hopefully make a profit.  To
that end, in the capitalist model, competition serves to lower prices
and increase quality to levels that the average consumer finds
acceptable.

A monopoly or duopoly environment distorts that; in a market with a
constrained number of providers, the conventional capitalistic model
can perform poorly or even fail entirely - as an example, consider
the LCD price fixing scandal last decade, where prices ended up
artificially high.

The current situation is worse; the telcos and cablecos have a bunch
of incentives to prevent cannibalizing their existing profitable 
pay TV product lines... which are seeing competition from the likes 
of Netflix.  And there's even some legitimacy there:  if all of those 
customers suddenly dropped their pay TV service and went to Netflix,
the whole economic underpinnings of a cable TV company could be
thrown into disarray.  Because those pay TV subscribers are in some 
way contributing to covering the opex and capex of the cable TV 
distribution network.  That'd also be damaging to the last mile IP
connectivity, heh.

But it's hard to have an honest discussion about all of this when those
involved are so busy trying to spin things in their favor, and to keep
the status quo, etc.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

2014-05-15 Thread Blake Dunlap

I agree, and those peers should be then paid for the bits that your
customers are requesting that they send through you if you cannot
maintain a balanced peer relationship with them. It's shameful that
access networks are attempting to not pay for their leeching of mass
amounts of data in clear violation of standard expectations for
balanced peering agreements.

Oh... you meant something else?

-Blake

On Thu, May 15, 2014 at 12:34 PM, Livingood, Jason
jason_living...@cable.comcast.com wrote:
 On 5/15/14, 1:28 PM, Nick B n...@pelagiris.orgmailto:n...@pelagiris.org 
 wrote:

 By categorically untrue do you mean FCC's open internet rules allow us to 
 refuse to upgrade full peers?

 Throttling is taking, say, a link from 10G and applying policy to constrain 
 it to 1G, for example. What if a peer wants to go from a balanced 
 relationship to 10,000:1, well outside of the policy binding the 
 relationship? Should we just unquestionably toss out our published policy – 
 which is consistent with other networks – and ignore expectations for other 
 peers?

 Jason

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality


On May 15, 2014, at 10:18 AM, Jean-Francois Mezei jfmezei_na...@vaxination.ca 
wrote:

 On 14-05-15 10:26, Owen DeLong wrote:
 Choosing between Comcast and a legacy Telco is like choosing between 
 legionnaire’s disease and SARS.
 
 Twisted pair is certantly legacy.
 
 Is there a feeling that coax cable/DOSCIS is also legacy in terms of
 current capacity/speeds ? Or is that technology still considered viable
 against FTTH ?
 
 I realise that business practices make north american incumbents
 undesirable compared to the rest of the world, especially Comcast's
 dirty tricks with Netflix as an example.
 
 But in terms of the last mile technology and wiring (for instance, homes
 per HFC node) sre north american cavlecos up to par with the rest of the
 world ?

I am not speaking specifically about any one company here.

In North America, very few places have any level of FTTH. If you are in a rural 
area with USF subsidies, you are more likely to have FTTH than many urban 
areas. Co-ax, or if you’re somewhat lucky, HFC is about the best last mile 
technology available to most US subscribers.

In states where some city invested in municipal FTTH on an open-access basis, 
the incumbent $CABLECOS and $TELCOS have fought hard to push legislation making 
it illegal for other cities in the state to do the same.

The state of broadband networks in the US in general can best be described as 
pathetic and/or apathetic when it comes to the consumer’s interest. Lilly 
Tomlin summed this up very well in a number of her early comedy sketches where 
she pretended to be a telephone company operator. Her catch phrase was “We 
don’t care. We don’t have to. We’re the phone company!”

Further, it appears that several of the $CABLECOS and $TELCOS will actually 
attempt to quash their more vocal opponents by discussing public comments they 
make on a personal basis with said opponents employer and using them as a 
“negotiating tactic”. Personally, I think this is one of the most underhanded 
and lowest forms of an act of desperation to try and squash public debate. To 
be very clear… This statement is absolutely not targeted at any one company. 
There were several.

Owen

Re: Observations of an Internet Middleman (Level3)

That link is broken and insists that I install a windows upgrade for Flash on 
my Mac.

Owen

On May 15, 2014, at 10:17 AM, Paul Ferguson fergdawgs...@mykolab.com wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256
 
 On 5/15/2014 10:06 AM, Ryan Brooks wrote:
 
 It's a shame the use of 'fast lane' is ubiquitous in this argument.
 If the local distribution networks would like to actually build 
 something fast, then this would be a different story.
 
 Okay, then call it the faster lane or the uncongested lane or
 something that actually reflects bias and preferential treatment. It's
 a done deal now:
 
 http://www.washingtonpost.com/blogs/the-switch/wp/2014/05/15/fcc-approves-plan-to-allow-for-paid-priority-on-internet
 
 FYI,
 
 - - ferg
 
 
 - -- 
 Paul Ferguson
 VP Threat Intelligence, IID
 PGP Public Key ID: 0x54DC85B2
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v2.0.22 (MingW32)
 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
 
 iF4EAREIAAYFAlN09roACgkQKJasdVTchbJPlgEAtBpp0TKNZcIdrDkVP75Tni7a
 O9PvcnKZdaPNuNUpOb0A/RQ5hvrqPAu/QLSp8dPbcDSO5Zad8Z3JG67UfI6yaeJH
 =DhnX
 -END PGP SIGNATURE-

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Paul Ferguson

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

No  idea -- I use NoScript and block Flash (as well as other dangerous
 annoying embedded content) and it works for me.

- - ferg


On 5/15/2014 11:31 AM, Owen DeLong wrote:

 That link is broken and insists that I install a windows upgrade
 for Flash on my Mac.
 
 Owen
 
 On May 15, 2014, at 10:17 AM, Paul Ferguson
 fergdawgs...@mykolab.com wrote:
 
 On 5/15/2014 10:06 AM, Ryan Brooks wrote:
 
 It's a shame the use of 'fast lane' is ubiquitous in this
 argument. If the local distribution networks would like to
 actually build something fast, then this would be a different
 story.
 
 Okay, then call it the faster lane or the uncongested lane or 
 something that actually reflects bias and preferential treatment.
 It's a done deal now:
 
 http://www.washingtonpost.com/blogs/the-switch/wp/2014/05/15/fcc-approves-plan-to-allow-for-paid-priority-on-internet

  FYI,
 
 - ferg
 
 
 
 
 

- -- 
Paul Ferguson
VP Threat Intelligence, IID
PGP Public Key ID: 0x54DC85B2
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlN1CUMACgkQKJasdVTchbJ+qQEA1PVA2pd4UhdA8onBV/2v/XC/
/hVje/whDjYhLWXMZroA/RlUxhJqQgc5erlnLXz5S2blywH1+e0d6ZtQ96srChne
=ychK
-END PGP SIGNATURE-

Re: Observations of an Internet Middleman (Level3) (was: RIP

 Throttling is taking, say, a link from 10G and applying policy to constrain=
  it to 1G, for example.

Throttling is also trying to cram 20G of traffic through that same 10G
link.

 What if a peer wants to go from a balanced relation=
 ship to 10,000:1, well outside of the policy binding the relationship?

What if you're running a 10G port at saturation in both directions and
you decide to stop accepting announcements from the peer on that port?
Now you have a 10,000:0 ratio.  Then what?

 Should we just unquestionably toss out our published policy =96 which is 
 consis=
 tent with other networks =96 and ignore expectations for other peers?

What's your goal at the end of the day?

You have customers who are paying you for connectivity to Teh Interwebz.

Do you have an obligation to run a dedicated 100GbE to each and every
host on the planet?  No.

Do you have an obligation to make a reasonable effort to move the 
traffic that your customer is paying you for?  Yes.

At the end of the day, if I'm your customer and I'm trying to pull
50Mbps of data on my 50Mbps connection that I am buying from you, 
then it seems like a reasonable thing to expect that you'll have
the 50Mbps of capacity to actually fulfill the demand.  That does
not mean that I will actually GET 50Mbps - it just means that you
should be making a reasonable effort and especially that you are 
not actively sabotaging it, by aggregating it through a congested 
10Gbps port, or forcing the packets through a congested peer, or
any of a number of other underhanded things.  

If you cannot figure out how to arrange your transit and peering
affairs in a manner that allows you to deliver on what you've sold
to customers in the current unregulated model, I think you'll find
that the alternative of regulation is very much less palatable.

So, to answer your question, yes, if you're unable to figure out 
that Netflix is always going to generate tons more traffic than it
receives, and that your customers desperately want to get good
connectivity to there, then that's dumb.  Perhaps you should 
figure out how to arrange peering with sites where there's 
obviously going to be an unrectifiable traffic imbalance.

You're a service provider.  What should your goal be?  I would
have thought it obvious:  Provide the service.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Scott Weeks

From: Paul Ferguson fergdawgs...@mykolab.com
On 5/15/2014 10:06 AM, Ryan Brooks wrote:

 It's a shame the use of 'fast lane' is ubiquitous in this argument.
  If the local distribution networks would like to actually build 
 something fast, then this would be a different story.

Okay, then call it the faster lane or the uncongested lane or
something that actually reflects bias and preferential treatment. It's
a done deal now:

http://www.washingtonpost.com/blogs/the-switch/wp/2014/05/15/fcc-approves-plan-to-allow-for-paid-priority-on-internet




According to the doc it's not a done deal: 

The proposal is not a final rule, but the vote on Thursday is a
significant step forward

Agencies almost always change their rules from the initial proposal

The next phase will be four months of public comments, after which 
the commissioners will vote again on redrafted rules that are meant 
to take into account public opinion.

But, yeah.  I don't believe for a minute that all/any this is above 
board when it's coming from a lobbyist.

scott

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

2014-05-15 Thread Scott Helms

AFAIK Comcast wasn't consuming, mass amounts of data from Level 3
(Netflix's transit to them).  Are you implying that a retail customer has a
similar expectation (or should) as a tier 1 ISP has for peering?  I hope
not, that would be hyperbole verging on the silly.  Retail customer
agreement spell out, in every example I've seen, realistic terms and
expectations for service and those are very different from peering
arrangements.


Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

http://twitter.com/kscotthelms



On Thu, May 15, 2014 at 2:28 PM, Blake Dunlap iki...@gmail.com wrote:

 I agree, and those peers should be then paid for the bits that your
 customers are requesting that they send through you if you cannot
 maintain a balanced peer relationship with them. It's shameful that
 access networks are attempting to not pay for their leeching of mass
 amounts of data in clear violation of standard expectations for
 balanced peering agreements.

 Oh... you meant something else?

 -Blake

 On Thu, May 15, 2014 at 12:34 PM, Livingood, Jason
 jason_living...@cable.comcast.com wrote:
  On 5/15/14, 1:28 PM, Nick B n...@pelagiris.orgmailto:
 n...@pelagiris.org wrote:
 
  By categorically untrue do you mean FCC's open internet rules allow
 us to refuse to upgrade full peers?
 
  Throttling is taking, say, a link from 10G and applying policy to
 constrain it to 1G, for example. What if a peer wants to go from a balanced
 relationship to 10,000:1, well outside of the policy binding the
 relationship? Should we just unquestionably toss out our published policy –
 which is consistent with other networks – and ignore expectations for other
 peers?
 
  Jason

Re: Observations of an Internet Middleman (Level3) (was: RIP Network Neutrality

2014-05-15 Thread Jerry Dent

If traffic is unbalanced, what determines who is the payer and who is the
payee? Apparently whoever can hold on to their customers better while
performance is shit.


On Thu, May 15, 2014 at 1:28 PM, Blake Dunlap iki...@gmail.com wrote:

 I agree, and those peers should be then paid for the bits that your
 customers are requesting that they send through you if you cannot
 maintain a balanced peer relationship with them. It's shameful that
 access networks are attempting to not pay for their leeching of mass
 amounts of data in clear violation of standard expectations for
 balanced peering agreements.

 Oh... you meant something else?

 -Blake

 On Thu, May 15, 2014 at 12:34 PM, Livingood, Jason
 jason_living...@cable.comcast.com wrote:
  On 5/15/14, 1:28 PM, Nick B n...@pelagiris.orgmailto:
 n...@pelagiris.org wrote:
 
  By categorically untrue do you mean FCC's open internet rules allow
 us to refuse to upgrade full peers?
 
  Throttling is taking, say, a link from 10G and applying policy to
 constrain it to 1G, for example. What if a peer wants to go from a balanced
 relationship to 10,000:1, well outside of the policy binding the
 relationship? Should we just unquestionably toss out our published policy –
 which is consistent with other networks – and ignore expectations for other
 peers?
 
  Jason

Re: Observations of an Internet Middleman (Level3)

2014-05-15 Thread Christopher Morrow

On Thu, May 15, 2014 at 2:06 PM, Scott Helms khe...@zcorum.com wrote:
 Chris,

 You're not reading what I said, nor did I make a statement anything like
 one of the silly things you referenced (640k ram etc).  Prioritization isn't

yes I made a joke. (*three of them actually)

 that complex and today we handle the maximum amount of complexity already
 since everything is the same priority right now.

sure... simple networking, no priorities.

 You're trying to make the statement that giving multiple content providers
 priority somehow makes connectivity unworkable for consumers as if we don't
 have this problem already.  Consumers can easily starve themselves of

not unworkable for the consumer, per say. it makes guaranteeing that
'fast-lane' for those folk that do pay for it harder. The cableco/etc
will potentially have to provide the equivalent 'fast-lane' bandwidth
for each consumer, or risk contract breach with some of their paying
'fast-lane' purchasers.

or that's sort of what it looks like to me... of course statistical
multiplexing and 'long tail' and other things probably mean this isn't
a 'happens to all households' problem, but it could happen to a goodly
portion if enough services become popular in an example household.

 bandwidth with video or any other content and almost no connections in the
 US have any sort of intelligent fair usage buffering provided by the service
 provider.  This is true for both cable, telco, and other operators.

sure, but there's no contractual problem with lost bits/streams
today... because 'moviecompany' didn't pay for a 'premium service' (or
priority or...) from 'cableco'.

-chris


 Scott Helms
 Vice President of Technology
 ZCorum
 (678) 507-5000
 
 http://twitter.com/kscotthelms
 


 On Thu, May 15, 2014 at 2:01 PM, Christopher Morrow
 morrowc.li...@gmail.com wrote:

 On Thu, May 15, 2014 at 1:48 PM, Scott Helms khe...@zcorum.com wrote:
  Its not really that complex, if you think about it having 1s of
  'movieco' with the same priority is the status quo.  At the end of the
  day
  the QoS mechanics in DOCSIS are pretty straightforward and rely on
  service
  flows, while service flows can have equal priority I doubt most
  operators
  will sell more than a few (perhaps just one) top priority in a given a
  category.
 

 yes, there will only ever be 5 computers. or you couldn't possibly
 need more than 640kb of ram. or more than 4billion 'ip addresses'.

 I don't think you have to get to more than 10 or 20 of the stated
 examples before things get dicey ... Once a set of customers
 experience (and can measure) the effect, they'll back their complaints
 up to 'moviecompany' and some set of contract penalties will kick in,
 I suspect.

 Sure, if there is only one it's not a problem, but there are already
 not just one...

 
  Scott Helms
  Vice President of Technology
  ZCorum
  (678) 507-5000
  
  http://twitter.com/kscotthelms
  
 
 
  On Thu, May 15, 2014 at 1:22 PM, Christopher Morrow
  morrowc.li...@gmail.com wrote:
 
  On Thu, May 15, 2014 at 1:06 PM, Ryan Brooks r...@hack.net wrote:
   On 5/15/14, 11:58 AM, Joe Greco wrote:
  
   2) Netflix purchases 5Mbps fast lane
  
  
   I appreciate Joe's use of quotation marks here.A lot of the
   dialog
   has
   included this 'fast lane' terminology, yet all of us know there's no
   'fast
   lane' being constructed, rather just varying degrees of _slow_
   applied
   to
   existing traffic.
  
 
  please correct me if I'm wrong, but 'fast lane' really is (in this
  example):
'cableco' port from 'moviecompany' has 'qos' marking configuration
  to set all 'moviecompany' traffic (from this port!) to some priority
  level.
 
customer-port to 'cableco' has 'qos' handling/queuing that will
  ensure '5mbps' of 'moviecompany' is always going to get down the link
  to the customer, regardless of the other traffic the customer is
  requesting.
 
  right? (presume that in the rest of the 'cableco' network is
  protecting 'moviecompany' traffic as well, of course)
 
  So, when there are 1 'moviecompany' things to prioritize and deliver
  that's cool... but what about when there are 10? 100? 1000? doesn't
  the queuing get complicated? what if the 'cableco' customer with
  10mbps link has 3 people in the location all streaming from 3
  different 'moviecompany' organizations which have paid for 'fastlane'
  services?
 
  3 x 5 == 15 ... not 10. How will 'cableco' manage this when their
  100gbps inter-metro links are seeing +100gbps if 'fastlane' traffic
  and 'fastlane' traffic can't make it to the local metro from the
  remote one?
 
  This all seems much, much more complicated and expensive than just
  building out networking, which they will have to do in the end anyway,
  right? Only with 'fastlanes' there's extra capacity management and
  configuration and testing and ... all

Re: Observations of an Internet Middleman (Level3) (was: RIP

 So by extension, if you enter an agreement and promise to remain balanced y=
 ou can just willfully throw that out and abuse the heck out of it? Where do=
 es it end? Why even bother having peering policies at all then?

It doesn't strike you as a ridiculous promise to extract from someone?

Hi I'm an Internet company.  I don't actually know what the next big
thing next year will be but I promise that I won't host it on my network 
and cause our traffic to become lopsided.

Wow.  Is that what you're saying?

 To use an analogy, if you and I agree to buy a car together and agree to sw=
 itch off who uses it every other day, can I just say forget our agreement =
 =96 I=92m just going to drive the car myself every single day =96 its all m=
 ine=94?

Seems like a poor analogy since I'm pretty sure both parties on a peering
can use the port at the same time.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Observations of an Internet Middleman (Level3)

 That link is broken and insists that I install a windows upgrade for =
 Flash on my Mac.

Try

http://arstechnica.com/tech-policy/2014/05/fcc-votes-for-internet-fast-lanes-but-could-change-its-mind-later/


... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Observations of an Internet Middleman (Level3) (was: RIP