Re: Elephant in the room - Akamai

[Mark Tinka]

On 8/Dec/19 02:19, Jared Mauch wrote:

> I’ve been working hard to provide value to our AANP partners as well.  I’ll 
> try to stop responding to the list at this point but don’t hesitate to 
> contact me here or via other means if you’re seeing something weird.  I know 
> I resolved a problem a few days ago for someone quickly as there was a 
> misconfiguration left around.. We all make mistakes and can all do better.

Problems are part of the gig - otherwise we'd have no reason to get up
in the morning.

What matters is that there is someone you can find to help you fix them.
That's what makes all the difference.

So kudos to you, Jared, and the entire team out there at Akamai.

Mark.

Re: Elephant in the room - Akamai

[Jared Mauch]

On Dec 7, 2019, at 5:34 PM, Rod Beck  wrote:
> 
> Have there been any fundamental change in their network architecture that 
> might explain pulling these caches?


Please see my email on Friday where I outlined a few of the dynamics at play.  
Akamai isn’t just one thing, it’s an entire basket of products that all have 
their own resulting behaviors.  This is why even though you may peer with us 
directly you may not see 100% of the traffic from that interconnection.  (Take 
SSL for example, it’s often not served via the clusters in an ISP due to the 
security requirements we place on those racks, and this is something we treat 
very seriously!)

This is why I’m encouraging people to ping me off-list, because the dynamics at 
play for one provider don’t match across the board.  I know we have thousands 
of distinct sites that each have their own attributes and composition at play.

I’ve been working hard to provide value to our AANP partners as well.  I’ll try 
to stop responding to the list at this point but don’t hesitate to contact me 
here or via other means if you’re seeing something weird.  I know I resolved a 
problem a few days ago for someone quickly as there was a misconfiguration left 
around.. We all make mistakes and can all do better.

- jared

https://www.peeringdb.com/asn/20940

network billing software

[Mehmet Akcin]

hi there,

as a small isp network billing is done right is quite important. I do not
want to reinvent the wheel though.

I have been using librenms and I do think I need a tool that can be more
automated and more turn-key. I have looked into cacti, and also Ubersmith.
I wanted to check in here and ask what people are using to see if I can
find something more commonly known.

I would like as much as automation possible (auto email of the bill, etc.)
and if it can integrate with quickbooks even better. 95th percentile
billing is a must. let me know what your recommendations are.

mehmet

Re: Elephant in the room - Akamai

[Mark Tinka]

On 6/Dec/19 21:29, Keenan Tims wrote:

>
> We're currently seeing about 80% of our AS20940 origin traffic coming
> from transit, and I'm certain there's a significant additional amount
> which is difficult to identify coming from on-net caches at our
> upstream providers (though it appears from the thread that may be
> reducing as well). Only about 20% is coming from peering where we have
> significantly more capacity and lower costs. Whatever the algorithm is
> doing, from my perspective it doesn't make a lot of sense and is
> pretty frustrating, and I'm somewhat concerned about busting commits
> and possibly running into congestion for the next big event that does
> hit us, which would not be a problem if it were delivered over peering.

We've had 2 or 3 customers, in the last 3 months, complain about the
same thing - where they are seeing Akamai traffic drop over peering but
preferred via their transit service with us. We run a number of Akamai
AANP caches across our backbone.

We are working very closely with Akamai - and the customers - to resolve
this, I'll add.

Mark.

Re: Elephant in the room - Akamai

[Rod Beck]

Have there been any fundamental change in their network architecture that might 
explain pulling these caches?


From: NANOG  on behalf of Shawn L via NANOG 

Sent: Saturday, December 7, 2019 8:20 PM
To: Jared Mauch 
Cc: nanog@nanog.org 
Subject: Re: Elephant in the room - Akamai


Same -- we had an Akamai cache for 15+ years.  Then we were notified that it 
was done and were sent boxes to pack our stuff up and send it back.





-Original Message-
From: "Jared Mauch" 
Sent: Saturday, December 7, 2019 2:05pm
To: "Seth Mattinen" 
Cc: nanog@nanog.org
Subject: Re: Elephant in the room - Akamai



> On Dec 7, 2019, at 12:06 PM, Seth Mattinen  wrote:
>
> On 12/6/19 06:46, Fawcett, Nick via NANOG wrote:
>> We had three onsite Akamai caches a few months ago. They called us up and 
>> said they are removing that service and sent us boxes to pack up the 
>> hardware and ship back. We’ve had quite the increase in DIA traffic as a 
>> result of it.
>
>
> Same here, removed last month, and no more Akamai traffic over peering since.

This last part doesn’t sound right.

Can you send me details in private?

Thanks,

- Jared

Re: virginia beach

[Mark Tinka]

Indeed.

Even if the PFE and SLTE are side-by-side inside a room at the CLS, the
physical connection between them typically happens inside the BMH (beach
manhole).

So provided you can run the Management traffic between the SLTE and PFE
optically, you can separate both devices as far as your optics can carry
the full channel payload toward the wet plant. That could be anywhere
between 80km - 120km, depending on your optics and Tx launch power
requirements.

Mark.

On 6/Dec/19 16:04, Mehmet Akcin wrote:
> Thank you
>
> I know SLTE can be further than 100m
>
> On Fri, Dec 6, 2019 at 06:26 Mark Tinka  > wrote:
>
> If the SLTE and PFE are not going to be co-located next to each
> other, the 2 main considerations are:
>
>   * The Management signal between the SLTE and the PFE, which is
> generally ran over copper. So 100m becomes the distance limit.
> However, you can mitigate this by doing media conversion
> between optical and copper. Also, it's quite possible that
> modern PFE's and SLTE's can run Management traffic over
> optical signals. One would have to check with the vendors.
>
>   * The SLTE Launch Tx power into the wet plant toward the 1st
> repeater. You'd have to design this for every specific case;
> whether you want to co-locate the SLTE and PFE, or if you want
> to separate them. This is critical so that all payload
> channels are correctly balanced.
>
> Mark.
>
>
> On 5/Dec/19 05:53, Mehmet Akcin wrote:
>> is there any limitation of where an SLTE can be placed in terms
>> of distance from PFE?
>>
>> I have looked in to usual palaces and i was unable to confirm
>> there is any requirements for any distance. Any cons can you guys
>> think of that you want to share would be appreciated.
>>
>> here are places I have looked at
>>
>> 
>> http://opticalcloudinfra.com/index.php/2018/03/28/whys-hows-open-subsea-cable-systems/
>>
>> 
>> https://www.osa.org/osaorg/media/osa.media/OSAF/Banners/Cable_Powering.pdf
>>
>> 
>> http://opticalcloudinfra.com/index.php/2016/02/27/why-open-subsea-cable-systems/
>>
>> thank you
>>
>> On Tue, Nov 26, 2019 at 1:03 PM Mark Tinka > > wrote:
>>
>>
>>
>> On 26/Nov/19 18:47, Ben Cannon wrote:
>>
>> > Nowadays however, the CLS is looked at more like an ILA
>> shelter, or
>> > when feasible cable landings are going directly into metro
>> CNDCs as
>> > the physical gear is getting smaller and smaller and more
>> suitable for
>> > colocation.
>>
>> Nowadays, the SLTE sits in a proper data centre.
>>
>>
>> >
>> > There’s still the small matter of 11,000volts of power or more…
>> >  That’s usually kept out towards the sea.
>>
>> Yes, PFE's will generally live at the CLS.
>>
>> Mark.
>>
>
> -- 
> Mehmet
> +1-424-298-1903

Re: Elephant in the room - Akamai

[Jared Mauch]

> On Dec 7, 2019, at 3:01 PM, Eric Kuhnke  wrote:
> 
> I think this thread might be a perfect example that when an organization 
> reaches a sufficiently large size, one part of its engineering/operations 
> team may no longer be fully aware of what other work groups are doing. 
> Definitely a structural challenge for ISPs that span very large geographical 
> areas and services/roles. 

We are a decent sized (public) company.  You can look at the # of employees if 
you are curious.  I am but one person who can try to influence things.  I’ll 
say that if you have a few servers from us, it’s not going to serve the entire 
content set that our customers have.

I do remain open to look at your individual cases and see what can be done to 
improve though.  The answer may be nothing, but an e-mail also costs you 
little.  I’ve got several people I’m corresponding with and will continue to do 
so at least until the paychecks dry up.  Given the number of questions I still 
field about my prior employer, it may even last beyond that point :-)

- Jared

Re: Elephant in the room - Akamai

[Eric Kuhnke]

I think this thread might be a perfect example that when an organization
reaches a sufficiently large size, one part of its engineering/operations
team may no longer be fully aware of what other work groups are doing.
Definitely a structural challenge for ISPs that span very large
geographical areas and services/roles.





On Sat, Dec 7, 2019 at 11:06 AM Jared Mauch  wrote:

>
>
> > On Dec 7, 2019, at 12:06 PM, Seth Mattinen  wrote:
> >
> > On 12/6/19 06:46, Fawcett, Nick via NANOG wrote:
> >> We had three onsite Akamai caches a few months ago.  They called us up
> and said they are removing that service and sent us boxes to pack up the
> hardware and ship back.  We’ve had quite the increase in DIA traffic as a
> result of it.
> >
> >
> > Same here, removed last month, and no more Akamai traffic over peering
> since.
>
> This last part doesn’t sound right.
>
> Can you send me details in private?
>
> Thanks,
>
> - Jared

Re: Elephant in the room - Akamai

[Shawn L via NANOG]

Same -- we had an Akamai cache for 15+ years.  Then we were notified that it 
was done and were sent boxes to pack our stuff up and send it back.
 
 
-Original Message-
From: "Jared Mauch" 
Sent: Saturday, December 7, 2019 2:05pm
To: "Seth Mattinen" 
Cc: nanog@nanog.org
Subject: Re: Elephant in the room - Akamai




> On Dec 7, 2019, at 12:06 PM, Seth Mattinen  wrote:
> 
> On 12/6/19 06:46, Fawcett, Nick via NANOG wrote:
>> We had three onsite Akamai caches a few months ago. They called us up and 
>> said they are removing that service and sent us boxes to pack up the 
>> hardware and ship back. We’ve had quite the increase in DIA traffic as a 
>> result of it.
> 
> 
> Same here, removed last month, and no more Akamai traffic over peering since.

This last part doesn’t sound right.

Can you send me details in private?

Thanks,

- Jared

Re: Elephant in the room - Akamai

[Jared Mauch]

> On Dec 7, 2019, at 12:06 PM, Seth Mattinen  wrote:
> 
> On 12/6/19 06:46, Fawcett, Nick via NANOG wrote:
>> We had three onsite Akamai caches a few months ago.  They called us up and 
>> said they are removing that service and sent us boxes to pack up the 
>> hardware and ship back.  We’ve had quite the increase in DIA traffic as a 
>> result of it.
> 
> 
> Same here, removed last month, and no more Akamai traffic over peering since.

This last part doesn’t sound right.

Can you send me details in private?

Thanks,

- Jared

Re: Elephant in the room - Akamai

[Seth Mattinen]

On 12/6/19 06:46, Fawcett, Nick via NANOG wrote:
We had three onsite Akamai caches a few months ago.  They called us up 
and said they are removing that service and sent us boxes to pack up the 
hardware and ship back.  We’ve had quite the increase in DIA traffic as 
a result of it.





Same here, removed last month, and no more Akamai traffic over peering 
since.

Re: Arista Switch Suggestion

[Mike Hammett]

*grumble* 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Siyuan Miao"  
To: "Mike Hammett"  
Cc: "Steve Meuse" , "nanog"  
Sent: Friday, December 6, 2019 6:00:13 PM 
Subject: Re: Arista Switch Suggestion 


Yes that's required. 


On Sat, Dec 7, 2019 at 7:05 AM Mike Hammett < na...@ics-il.net > wrote: 




Am I reading correctly in that there has to be a layer 3 configuration on the 
VLAN for that to function? 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 



From: "Siyuan Miao" < avel...@misaka.io > 
To: "Mike Hammett" < na...@ics-il.net > 
Cc: "Steve Meuse" < sme...@mara.org >, "nanog" < nanog@nanog.org > 
Sent: Friday, December 6, 2019 4:49:45 PM 
Subject: Re: Arista Switch Suggestion 


I can confirm 7050X series do support this feature.We're using 7050SX and 
7050S, 7050S isn't supported 


https://www.arista.com/en/um-eos/eos-section-11-6-ethernet-configuration-commands#ww1310058
 











On Sat, Dec 7, 2019 at 6:42 AM Mike Hammett < na...@ics-il.net > wrote: 




I have not found x-flow to have the accuracy I would like. Either there's a 
loss of information due to sampling and such low-usage interfaces (or VLANs in 
this case) are lost in the noise or there's information overload due to no 
sampling at all. 




I have seen very few platforms expose counter information about VLANs in the 
same way they do regular interfaces. Some Juniper platforms, Mikrotik, and I 
hear some Aristas as well. 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 



From: "Steve Meuse" < sme...@mara.org > 
To: "Mike Hammett" < na...@ics-il.net > 
Cc: "nanog" < nanog@nanog.org > 
Sent: Friday, December 6, 2019 4:20:08 PM 
Subject: Re: Arista Switch Suggestion 




You should be able to do that with Sflow, which they all/most support. 


Also, this seems like standard Ifmib stuff, any snmp poller should be able to 
handle that, from a metrics perspective . 



-Steve 




On Fri, Dec 6, 2019 at 4:31 PM Mike Hammett < na...@ics-il.net > wrote: 




I asked over at https://puck.nether.net/mailman/listinfo/arista-nsp a couple 
weeks ago, but didn't get an answer, so I have moved to a larger group. 



I understand that some Arista switches will expose each VLAN in SNMP so I can 
monitor traffic on a VLAN independently of over VLANs on that same physical 
interface. Some of them don't. 


Which ones do? 


I prefer a solid used switch. 




10G ports are fine. 





- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

Re: Anyone have contacts at Bharti Airtel?

[Anurag Bhatia]

I have friends at AS9498. Contacting you offlist to put you in touch with
them.

On Sat, Dec 7, 2019 at 4:54 PM Suresh Ramasubramanian 
wrote:

> Post on sa...@sanog.org there should be some Airtel people there
>
> @anurag can you please forward to someone there
>
> --srs
>
> --
> *From:* NANOG  on behalf of Elmar K. Bins <
> e...@4ever.de>
> *Sent:* Saturday, December 7, 2019 3:40 PM
> *To:* Bottiger
> *Cc:* nanog@nanog.org
> *Subject:* Re: Anyone have contacts at Bharti Airtel?
>
> bottige...@gmail.com (Bottiger) wrote:
>
> > Does anyone have any contacts at Bharti Airtel? I either get no response
> or
> > full inbox for emails in their WHOIS at AS9498 and AS24560.
>
> Hi, if you get a response, please share...I'm also at a loss there...
>
> Elmar.
>


-- 


Anurag Bhatia
anuragbhatia.com

Re: Anyone have contacts at Bharti Airtel?

[Suresh Ramasubramanian]

Post on sa...@sanog.org there should be some Airtel people there

@anurag can you please forward to someone there

--srs


From: NANOG  on behalf of Elmar K. Bins 
Sent: Saturday, December 7, 2019 3:40 PM
To: Bottiger
Cc: nanog@nanog.org
Subject: Re: Anyone have contacts at Bharti Airtel?

bottige...@gmail.com (Bottiger) wrote:

> Does anyone have any contacts at Bharti Airtel? I either get no response or
> full inbox for emails in their WHOIS at AS9498 and AS24560.

Hi, if you get a response, please share...I'm also at a loss there...

Elmar.

Re: Anyone have contacts at Bharti Airtel?

[Elmar K. Bins]

bottige...@gmail.com (Bottiger) wrote:

> Does anyone have any contacts at Bharti Airtel? I either get no response or
> full inbox for emails in their WHOIS at AS9498 and AS24560.

Hi, if you get a response, please share...I'm also at a loss there...

Elmar.