RE: CGNAT scaling cost (was Re: V6 still not supported)

[Vasilenko Eduard via NANOG]

No.
I have already forgotten that SDH did exist (and yes, I remember X.25 - I have 
operated X.25 network).
I was talking in the next message about 100GE.
In fact, the situation would be similar for 10E too.

Ed/
-Original Message-
From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org] On 
Behalf Of Masataka Ohta
Sent: Thursday, March 31, 2022 3:56 AM
To: nanog@nanog.org
Subject: Re: CGNAT scaling cost (was Re: V6 still not supported)

Vasilenko Eduard via NANOG wrote:

> CGNAT cost was very close to 3x compared to routers of the same 
> performance.

That should be because you are comparing cost of carrier, that is telco, grade 
NAT and consumer grade routers.

Remember the cost of carrier grade datalink of SONET/SDH.

Masataka Ohta

Re: CGNAT scaling cost (was Re: V6 still not supported)

[Masataka Ohta]

Vasilenko Eduard via NANOG wrote:


CGNAT cost was very close to 3x compared to routers of the
same performance.


That should be because you are comparing cost of carrier,
that is telco, grade NAT and consumer grade routers.

Remember the cost of carrier grade datalink of SONET/SDH.

Masataka Ohta

RE: RE: CGNAT scaling cost (was V6 still not supported)

[Vasilenko Eduard via NANOG]

Hi Jared,
I did mean big systems where performance needed is n*100Gbps or bigger.
For router or CGNAT: the chassis cost is less than 1 card. Hence, all cost is 
in ports (for the big router up to 95% if counting QSFP too). Chassis, power 
supplies, switching fabrics - could be discarded for a big system cost 
estimation.
You could think that I was comparing the average cost for the 100GE port of the 
router and CGNAT
That may be wire-speed for the reasonable average packet size (750B?)
And typical profile 6:1 upstream/downstream.

Scaling router and especially CGNAT (that is very often big because 
centralized) means: adding cards to empty slots.
Where all cost is in the ports only.
It is a little more complex for CGNAT because input/output ports are separate 
from processing cards. But let's assume that the proper mix is inserted.

Of course, if you would use router card ports by 50% (or install not all 
processing cards for CGNAT) then the cost may vary.
But let's assume almost full utilization for comparable results. It would be 
the case for the big populated system anyway.

Hence, yes, it is almost linear for big systems.
But if you would start from just 1 card (not possible for a big system?) then 
the port cost would start from 2x (+ common components).

Eduard
-Original Message-
From: Jared Brown [mailto:nanog-...@mail.com] 
Sent: Wednesday, March 30, 2022 8:17 PM
To: Vasilenko Eduard 
Cc: nanog@nanog.org
Subject: Re: RE: CGNAT scaling cost (was V6 still not supported)

Hi Eduard,

Do I interpret your findings correctly, if this means that CGNAT costs scale 
more or less linearly with traffic growth over time?

And as a corollary, that the cost of scaling CGNAT in itself isn't likely a 
primary driver for IPv6 adoption?


- Jared


Vasilenko Eduard wrote:
>
> CGNAT cost was very close to 3x compared to routers of the same performance.
> Hence, 1 hop through CGNAT = 3 hops through routers.
> 3 router hops maybe the 50% of overall hops in the particular Carrier (or 
> even less).
>
> DWDM is 3x more expensive per hop. Fiber is much more expensive (greatly 
> varies per situation and distance).
> Hence, +50% for IP does not mean +50% for the whole infrastructure, not at 
> all.
>
> I was on all primary vendors for 2.5 decades. 3x cost of NAT was consistent 
> for all vendors and at all times.
> Because it is a "Network processor" (really flexible one with a big memory) 
> against "specialized ASIC". COTS (x86) is much worse for the big scale - does 
> not make sense to compare.
> It has started to decrease recently when SFPs have become the bigger part of 
> the router (up to 50% for single-mode).
> Hence, I expect the decrease of the difference between router and CGNAT cost 
> to 2x long-term.
> Optical vendors are more capable to protect their margins.
>
> It is a different situation in Mobile Carriers, where Packet Core and Gi-LAN 
> were never accelerated in hardware.
> Everything else is so expensive (x86) per Gbps, that CGNAT is not visible in 
> the cost.
>
> Eduard
> -Original Message-
> From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org] On 
> Behalf Of Jared Brown
> Sent: Wednesday, March 30, 2022 6:33 PM
> To: nanog@nanog.org
> Subject: CGNAT scaling cost (was Re: V6 still not supported)
>
> An oft-cited driver of IPv6 adoption is the cost of scaling CGNAT or 
> equivalent infrastructure for IPv4.
>
> Those of you facing costs for scaling CGNAT, are your per unit costs rising 
> or declining faster or slower than your IPv4 traffic growth?
>
> I ask because I realize I am not fit to evaluate the issue on a general 
> level, as, most probably due to our insignificant scale, our CGNAT marginal 
> costs are zero. This is mainly because our CGNAT solution is oversized to our 
> needs. Even though scaling up our currently oversized system further would 
> lower per unit costs, I understand this may not be the case outside our 
> bubble.
>
>
> - Jared
>

Re: RE: CGNAT scaling cost (was V6 still not supported)

[Jared Brown]

Hi Eduard,

Do I interpret your findings correctly, if this means that CGNAT costs scale 
more or less linearly with traffic growth over time?

And as a corollary, that the cost of scaling CGNAT in itself isn't likely a 
primary driver for IPv6 adoption?


- Jared


Vasilenko Eduard wrote:
>
> CGNAT cost was very close to 3x compared to routers of the same performance.
> Hence, 1 hop through CGNAT = 3 hops through routers.
> 3 router hops maybe the 50% of overall hops in the particular Carrier (or 
> even less).
>
> DWDM is 3x more expensive per hop. Fiber is much more expensive (greatly 
> varies per situation and distance).
> Hence, +50% for IP does not mean +50% for the whole infrastructure, not at 
> all.
>
> I was on all primary vendors for 2.5 decades. 3x cost of NAT was consistent 
> for all vendors and at all times.
> Because it is a "Network processor" (really flexible one with a big memory) 
> against "specialized ASIC". COTS (x86) is much worse for the big scale - does 
> not make sense to compare.
> It has started to decrease recently when SFPs have become the bigger part of 
> the router (up to 50% for single-mode).
> Hence, I expect the decrease of the difference between router and CGNAT cost 
> to 2x long-term.
> Optical vendors are more capable to protect their margins.
>
> It is a different situation in Mobile Carriers, where Packet Core and Gi-LAN 
> were never accelerated in hardware.
> Everything else is so expensive (x86) per Gbps, that CGNAT is not visible in 
> the cost.
>
> Eduard
> -Original Message-
> From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org] On 
> Behalf Of Jared Brown
> Sent: Wednesday, March 30, 2022 6:33 PM
> To: nanog@nanog.org
> Subject: CGNAT scaling cost (was Re: V6 still not supported)
>
> An oft-cited driver of IPv6 adoption is the cost of scaling CGNAT or 
> equivalent infrastructure for IPv4.
>
> Those of you facing costs for scaling CGNAT, are your per unit costs rising 
> or declining faster or slower than your IPv4 traffic growth?
>
> I ask because I realize I am not fit to evaluate the issue on a general 
> level, as, most probably due to our insignificant scale, our CGNAT marginal 
> costs are zero. This is mainly because our CGNAT solution is oversized to our 
> needs. Even though scaling up our currently oversized system further would 
> lower per unit costs, I understand this may not be the case outside our 
> bubble.
>
>
> - Jared
>

RE: CGNAT scaling cost (was Re: V6 still not supported)

[Vasilenko Eduard via NANOG]

CGNAT cost was very close to 3x compared to routers of the same performance.
Hence, 1 hop through CGNAT = 3 hops through routers.
3 router hops maybe the 50% of overall hops in the particular Carrier (or even 
less).

DWDM is 3x more expensive per hop. Fiber is much more expensive (greatly varies 
per situation and distance).
Hence, +50% for IP does not mean +50% for the whole infrastructure, not at all.

I was on all primary vendors for 2.5 decades. 3x cost of NAT was consistent for 
all vendors and at all times.
Because it is a "Network processor" (really flexible one with a big memory) 
against "specialized ASIC". COTS (x86) is much worse for the big scale - does 
not make sense to compare.
It has started to decrease recently when SFPs have become the bigger part of 
the router (up to 50% for single-mode).
Hence, I expect the decrease of the difference between router and CGNAT cost to 
2x long-term.
Optical vendors are more capable to protect their margins.

It is a different situation in Mobile Carriers, where Packet Core and Gi-LAN 
were never accelerated in hardware.
Everything else is so expensive (x86) per Gbps, that CGNAT is not visible in 
the cost.

Eduard
-Original Message-
From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org] On 
Behalf Of Jared Brown
Sent: Wednesday, March 30, 2022 6:33 PM
To: nanog@nanog.org
Subject: CGNAT scaling cost (was Re: V6 still not supported)

An oft-cited driver of IPv6 adoption is the cost of scaling CGNAT or equivalent 
infrastructure for IPv4.

Those of you facing costs for scaling CGNAT, are your per unit costs rising or 
declining faster or slower than your IPv4 traffic growth?

I ask because I realize I am not fit to evaluate the issue on a general level, 
as, most probably due to our insignificant scale, our CGNAT marginal costs are 
zero. This is mainly because our CGNAT solution is oversized to our needs. Even 
though scaling up our currently oversized system further would lower per unit 
costs, I understand this may not be the case outside our bubble.


- Jared

RE: CGNAT

[aaron1]

We thought about it for a while at the ISP where I work, and went with Juniper 
MX960's w/MS-MPC-128G.  Been working quite nice for us.

Initially, we went with smaller MX104 w/MS-MIC-16G to prove it out on our 
~4,000 lower bandwidth DSL customers... when convinced, we then went all in 
with multiple MX960's w/MS-MPC-128Gnow over 50,000 customers of dsl, cable 
modem and ftth

...all that behind about ~/21

I'll add that we already had the 960's for the 100gig mpls sp core we had 
built, so it was an investment only on the service module to do cgnat.


-Aaron

RE: CGNAT

[Tony Wicks]

While I won't go into the costs as well, I've got actual work to do I must say 
my calculations of purchase ipv4 (@25USD/IP) vs CGNAT have always fallen 
significantly into the CGNAT camp. If you are doing a stand alone A10 or 
similar yes things would be different. If you are already buying suitable BNG's 
however the additional cost of MS-MPC cards (Juniper) or ISA2/ESA (Nokia) is 
likely to be far less than the stand alone option. While the BNG services cards 
are not cheap they do "just work" as a solution and a 10 or 20 to one ratio is 
easily achievable. Nokia 7750's with ESA "cards" are a massively scalable 
option.



-Original Message-
From: NANOG  On Behalf Of Kevin Burke
Sent: Thursday, 4 March 2021 6:42 am
To: Jared Brown ; nanog@nanog.org
Subject: Re: CGNAT

Can you share your cost comparison?  

Re: CGNAT

[Kevin Burke]

Can you share your cost comparison?  

If I assume the IPv4 purchased addresses will be useful for the next 15+ years 
they do make a ton of sense.  Estimating the amount of traffic 5+ years from 
now is not something I have high confidence in.  Making predictions is hard, 
especially about the future.  

What kind of IPv4/IPv6 traffic ratio's should we expect 5-15 years from now?  I 
assume there is no simple answer for this.  

An ISP with mostly enterprise customer's would expect different assumptions 
from a mobile phone provider.  This may be one of those times where every 
answer is correct, just not for everyone.  The whole "one size fits some" kind 
of solution.  
 
Kevin Burke
802-540-0979
Burlington Telecom

200 Church St, Burlington, VT

On 3/1/21, 2:38 PM, "NANOG on behalf of Jared Brown" 
 wrote:

WARNING!! This message originated from an External Source. Please use 
proper judgment and caution when opening attachments, clicking links, or 
responding to this email.

Kevin,

One of the presented options isn't like the others. As such the comparison 
isn't really fair, especially if you expect to run your business longer than 7 
years.

If you buy more IPv4 space you will neither have to deal with CGNAT nor 
worry about traffic growth. Both of those benefits are easily worth the (short 
term) premium.

In the long term, buying more IPv4 blocks now is likely to be cheaper than 
running CGNAT for the foreseeable future.

To echo Owen, in general, the economics today still work out to make 
purchasing addresses more favorable than CGNAT.

- Jared


Sent: Tue Feb 2314:36:48 UTC 2021
From: Kevin Burke kburke at burlingtontelecom.com
To: nanog@nanog.org
    Subject: Re: CGNAT

We are looking at implementing a similar solution with A10 for CGNAT.

We've been in touch with A10. Just wondering if there are some alternative 
vendors that anyone would recommend. We'd probably be looking at a solution to 
support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting 
point. A solution that is as transparent to user experience as possible is a 
priority.


The numbers below are for a similar target of subscriber’s and peak 
bandwidth.

We assumed a couple of numbers:
Current Peak Bandwidth = 40G
Remaining IPv4 traffic after migration = 20% (Seen references to 10% or 20% 
on this forum)
Future Bandwidth Growth = 2x (no data behind this assumption)
Future CGNAT’ed bandwidth = 15Gbps
Equipment & budget lifecycle = 7Yr

Getting that data led us to this price comparison:

Solution
Lifecycle/ Term
Annual Cost/Sub
Product Lifecycle Cost/Sub
Lease IPv4 Cogent
7
$ 4.45
 $   31.13
A10 CGNAT 15Gb 7Yr
7
$ 1.21
 $ 8.47
A10 CGNAT 40Gb 7Yr
7
$ 1.95
 $   13.68
Purchase @ $25 7Yr
7
$ 3.57
 $   25.00


The current plan is implement an A10 CGNAT solution after upgrading our 
network for IPv6.  In the interim we will have to lease IPv4 to tide us over.

I would be curious to see what other’s estimate the costs of various 
approaches.  Feel free to ping me off-list for more specific numbers.

Kevin Burke
802-540-0979
Burlington Telecom
200 Church St, Burlington, VT

Re: CGNAT

[Jared Brown]

Kevin,

One of the presented options isn't like the others. As such the comparison 
isn't really fair, especially if you expect to run your business longer than 7 
years.

If you buy more IPv4 space you will neither have to deal with CGNAT nor worry 
about traffic growth. Both of those benefits are easily worth the (short term) 
premium.

In the long term, buying more IPv4 blocks now is likely to be cheaper than 
running CGNAT for the foreseeable future.

To echo Owen, in general, the economics today still work out to make purchasing 
addresses more favorable than CGNAT.

- Jared


Sent: Tue Feb 2314:36:48 UTC 2021
From: Kevin Burke kburke at burlingtontelecom.com 
To: nanog@nanog.org
Subject: Re: CGNAT

We are looking at implementing a similar solution with A10 for CGNAT.

We've been in touch with A10. Just wondering if there are some alternative 
vendors that anyone would recommend. We'd probably be looking at a solution to 
support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting 
point. A solution that is as transparent to user experience as possible is a 
priority.


The numbers below are for a similar target of subscriber’s and peak bandwidth.

We assumed a couple of numbers:
Current Peak Bandwidth = 40G
Remaining IPv4 traffic after migration = 20% (Seen references to 10% or 20% on 
this forum)
Future Bandwidth Growth = 2x (no data behind this assumption)
Future CGNAT’ed bandwidth = 15Gbps
Equipment & budget lifecycle = 7Yr

Getting that data led us to this price comparison:

Solution
Lifecycle/ Term
Annual Cost/Sub
Product Lifecycle Cost/Sub
Lease IPv4 Cogent
7
$ 4.45
 $   31.13
A10 CGNAT 15Gb 7Yr
7
$ 1.21
 $ 8.47
A10 CGNAT 40Gb 7Yr
7
$ 1.95
 $   13.68
Purchase @ $25 7Yr
7
$ 3.57
 $   25.00


The current plan is implement an A10 CGNAT solution after upgrading our network 
for IPv6.  In the interim we will have to lease IPv4 to tide us over.

I would be curious to see what other’s estimate the costs of various 
approaches.  Feel free to ping me off-list for more specific numbers.

Kevin Burke
802-540-0979
Burlington Telecom
200 Church St, Burlington, VT

Re: CGNAT

[JORDI PALET MARTINEZ via NANOG]

I did this "economics" exercise for a customer having 25.000.000 customers 
(DSL, GPON and cellular). Even updating/replacing the CPEs, the cost of 464XLAT 
deployment was cheaper than CGN or anything else.

Also, if you consider the cost of buying more IPv4 addresses instead of 
investing that money in CGN, you avoid CGN troubles (like black listening your 
IPv4 addresses by Sony and others and the consequently operation/management 
expenses to rotate IPv4 addresses in the CGN, resolve customers problems, 
etc.), it becomes cheaper than CGN boxes.

It's easy to predict that you will buy now CGN and tomorrow you will need to 
buy some new IPv4 addresses because that black listening.

Regards,
Jordi
@jordipalet
 
 

El 24/2/21 3:13, "NANOG en nombre de Owen DeLong via NANOG" 
 escribió:



> On Feb 22, 2021, at 6:44 AM, na...@jima.us wrote:
> 
> While I don't doubt the accuracy of Lee's presentation at the time, at 
least two base factors have changed since then:
> 
> - Greater deployment of IPv6 content (necessitating less CGN capacity per 
user)

This is only true if the ISP in question is implementing IPv6 along side 
their CGN deployment and only if they get a significant uptake of IPv6 
capability by their end users.

> - Increased price of Legacy IP space on the secondary market (changing 
the formula) -- strictly speaking, this presentation was still in "primary 
market" era for LACNIC/ARIN/AFRINIC

While that’s true, even at current prices, IPv4 addresses are cheaper to 
buy and/or lease than CGN.

> IPv6 migration is not generally aided by CGNAT, but CGNAT deployment is 
generally aided by IPv6 deployment; to reiterate the earlier point, any ISPs 
deploying CGNAT without first deploying IPv6 are burning cash.

Yep.

I still think that implementing CGN is a good way to burn cash vs. the 
alternatives, but YMMV.

Owen

> 
> - Jima
> 
> From: NANOG On Behalf Of Owen DeLong
> Sent: Sunday, February 21, 2021 16:59
    > To: Steve Saner
> Cc: nanog@nanog.org
> Subject: Re: CGNAT
> 
> 
> On Feb 18, 2021, at 8:38 AM, Steve Saner wrote:
> 
>> We are starting to look at CGNAT solutions. The primary motivation at 
the moment is to extend current IPv4 resources, but IPv6 migration is also a 
factor.
> 
> IPv6 Migration is generally not aided by CGNAT.
> 
> In general, the economics today still work out to make purchasing or 
leasing addresses more favorable than CGNAT.
> 
> It’s a bit dated by now, but still very relevant, see Lee Howard’s 
excellent research presented at the 2012 Rocky
> mountain v6 task force meeting:
> 
> https://www.rmv6tf.org/wp-content/uploads/2012/11/TCO-of-CGN1.pdf
> 
> Owen
> 
> 
> We've been in touch with A10. Just wondering if there are some 
alternative vendors that anyone would recommend. We'd probably be looking at a 
solution to support 5k to 15k customers and bandwidth up to around 30-40 gig as 
a starting point. A solution that is as transparent to user experience as 
possible is a priority.
> 
> Thanks
> 
> -- 
> Steve Saner
> ideatek HUMAN AT OUR VERY FIBER
> This email transmission, and any documents, files or previous email 
messages attached to it may contain confidential information. If the reader of 
this message is not the intended recipient or the employee or agent responsible 
for delivering the message to the intended recipient, you are hereby notified 
that any dissemination, distribution or copying of this communication is 
strictly prohibited. If you are not, or believe you may not be, the intended 
recipient, please advise the sender immediately by return email or by calling 
tel:620.543.5026. Then take all steps necessary to permanently delete the email 
and all attachments from your computer system.
> 




**
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or 
confidential. The information is intended to be for the exclusive use of the 
individual(s) named above and further non-explicilty authorized disclosure, 
copying, distribution or use of the contents of this information, even if 
partially, including attached files, is strictly prohibited and will be 
considered a criminal offense. If you are not the intended recipient be aware 
that any disclosure, copying, distribution or use of the contents of this 
information, even if partially, including attached files, is strictly 
prohibited, will be considered a criminal offense, so you must reply to the 
original sender to inform about this communication and delete it.

Re: CGNAT

[Owen DeLong via NANOG]

> On Feb 22, 2021, at 6:44 AM, na...@jima.us wrote:
> 
> While I don't doubt the accuracy of Lee's presentation at the time, at least 
> two base factors have changed since then:
> 
> - Greater deployment of IPv6 content (necessitating less CGN capacity per 
> user)

This is only true if the ISP in question is implementing IPv6 along side their 
CGN deployment and only if they get a significant uptake of IPv6 capability by 
their end users.

> - Increased price of Legacy IP space on the secondary market (changing the 
> formula) -- strictly speaking, this presentation was still in "primary 
> market" era for LACNIC/ARIN/AFRINIC

While that’s true, even at current prices, IPv4 addresses are cheaper to buy 
and/or lease than CGN.

> IPv6 migration is not generally aided by CGNAT, but CGNAT deployment is 
> generally aided by IPv6 deployment; to reiterate the earlier point, any ISPs 
> deploying CGNAT without first deploying IPv6 are burning cash.

Yep.

I still think that implementing CGN is a good way to burn cash vs. the 
alternatives, but YMMV.

Owen

> 
> - Jima
> 
> From: NANOG On Behalf Of Owen DeLong
> Sent: Sunday, February 21, 2021 16:59
> To: Steve Saner
> Cc: nanog@nanog.org
> Subject: Re: CGNAT
> 
> 
> On Feb 18, 2021, at 8:38 AM, Steve Saner wrote:
> 
>> We are starting to look at CGNAT solutions. The primary motivation at the 
>> moment is to extend current IPv4 resources, but IPv6 migration is also a 
>> factor.
> 
> IPv6 Migration is generally not aided by CGNAT.
> 
> In general, the economics today still work out to make purchasing or leasing 
> addresses more favorable than CGNAT.
> 
> It’s a bit dated by now, but still very relevant, see Lee Howard’s excellent 
> research presented at the 2012 Rocky
> mountain v6 task force meeting:
> 
> https://www.rmv6tf.org/wp-content/uploads/2012/11/TCO-of-CGN1.pdf
> 
> Owen
> 
> 
> We've been in touch with A10. Just wondering if there are some alternative 
> vendors that anyone would recommend. We'd probably be looking at a solution 
> to support 5k to 15k customers and bandwidth up to around 30-40 gig as a 
> starting point. A solution that is as transparent to user experience as 
> possible is a priority.
> 
> Thanks
> 
> -- 
> Steve Saner
> ideatek HUMAN AT OUR VERY FIBER
> This email transmission, and any documents, files or previous email messages 
> attached to it may contain confidential information. If the reader of this 
> message is not the intended recipient or the employee or agent responsible 
> for delivering the message to the intended recipient, you are hereby notified 
> that any dissemination, distribution or copying of this communication is 
> strictly prohibited. If you are not, or believe you may not be, the intended 
> recipient, please advise the sender immediately by return email or by calling 
> tel:620.543.5026. Then take all steps necessary to permanently delete the 
> email and all attachments from your computer system.
> 

Re: CGNAT

[Mark Andrews]

IPv4AAS will also work easily for any ISP on the planet.

CGNAT requires IPv4 address space between the CGNAT and the customer CPE which 
doesn’t overlap with that on the Internet nor that behind the CPE (no you can’t 
use RFC 1918).  100.64/10 gives you ~4M addresses which fit this criteria but 
that isn’t enough without reuse for the larger ISPs.

IPv4AAS uses no IPv4 addresses between the B4/NAT64/… and the CPE.

You should also be able to out source IPv4AAS to specialist providers.  There 
is no requirement to run your own hardware.  You just populate your 
DHCPv6/RA/IPV4ONLY.ARPA with the correct information and the traffic will be 
delivered to the specialist providers.  I’m not sure if there are any out there 
yet but it is a business opportunity for anyone that is already running such 
boxes.

Mark

> On 24 Feb 2021, at 09:43, Owen DeLong  wrote:
> 
> That’s provably not true if the IPv4AAS implementation is done carefully.
> 
> Owen
> 
> 
>> On Feb 19, 2021, at 12:11 PM, Tony Wicks  wrote:
>> 
>> Because then a large part of the Internet won't work
>> 
>> From: NANOG  on behalf of Mark 
>> Andrews 
>> Sent: Saturday, 20 February 2021, 9:04 am
>> To: Steve Saner
>> Cc: nanog@nanog.org
>> Subject: Re: CGNAT
>> 
>> Why not go whole hog and provide IPv4 as a service? That way you are not 
>> waiting for your customers to turn up IPv6 to take the load off your NAT box.
>> 
>> Yes, you can do it dual stack but you have waited so long you may as well 
>> miss that step along the deployment path.
>> -- 
>> Mark Andrews
>> 
>>> On 20 Feb 2021, at 01:55, Steve Saner  wrote:
>>> 
>>> 
>>> We are starting to look at CGNAT solutions. The primary motivation at the 
>>> moment is to extend current IPv4 resources, but IPv6 migration is also a 
>>> factor.
>>> 
>>> We've been in touch with A10. Just wondering if there are some alternative 
>>> vendors that anyone would recommend. We'd probably be looking at a solution 
>>> to support 5k to 15k customers and bandwidth up to around 30-40 gig as a 
>>> starting point. A solution that is as transparent to user experience as 
>>> possible is a priority.
>>> 
>>> Thanks
>>> 
>>> -- 
>>> Steve Saner
>>> ideatek HUMAN AT OUR VERY FIBER
>>> This email transmission, and any documents, files or previous email 
>>> messages attached to it may contain confidential information. If the reader 
>>> of this message is not the intended recipient or the employee or agent 
>>> responsible for delivering the message to the intended recipient, you are 
>>> hereby notified that any dissemination, distribution or copying of this 
>>> communication is strictly prohibited. If you are not, or believe you may 
>>> not be, the intended recipient, please advise the sender immediately by 
>>> return email or by calling 620.543.5026. Then take all steps necessary to 
>>> permanently delete the email and all attachments from your computer system.
>>> 
>> 
> 

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742  INTERNET: ma...@isc.org

Re: CGNAT

[Owen DeLong via NANOG]

That’s provably not true if the IPv4AAS implementation is done carefully.

Owen


> On Feb 19, 2021, at 12:11 PM, Tony Wicks  wrote:
> 
> Because then a large part of the Internet won't work
> 
> From: NANOG  on behalf of Mark 
> Andrews 
> Sent: Saturday, 20 February 2021, 9:04 am
> To: Steve Saner
> Cc: nanog@nanog.org
> Subject: Re: CGNAT
> 
> Why not go whole hog and provide IPv4 as a service? That way you are not 
> waiting for your customers to turn up IPv6 to take the load off your NAT box.
> 
> Yes, you can do it dual stack but you have waited so long you may as well 
> miss that step along the deployment path.
> -- 
> Mark Andrews
> 
>> On 20 Feb 2021, at 01:55, Steve Saner  wrote:
>> 
>> 
>> We are starting to look at CGNAT solutions. The primary motivation at the 
>> moment is to extend current IPv4 resources, but IPv6 migration is also a 
>> factor.
>> 
>> We've been in touch with A10. Just wondering if there are some alternative 
>> vendors that anyone would recommend. We'd probably be looking at a solution 
>> to support 5k to 15k customers and bandwidth up to around 30-40 gig as a 
>> starting point. A solution that is as transparent to user experience as 
>> possible is a priority.
>> 
>> Thanks
>> 
>> -- 
>> Steve Saner
>> ideatek HUMAN AT OUR VERY FIBER
>> This email transmission, and any documents, files or previous email messages 
>> attached to it may contain confidential information. If the reader of this 
>> message is not the intended recipient or the employee or agent responsible 
>> for delivering the message to the intended recipient, you are hereby 
>> notified that any dissemination, distribution or copying of this 
>> communication is strictly prohibited. If you are not, or believe you may not 
>> be, the intended recipient, please advise the sender immediately by return 
>> email or by calling 620.543.5026 . Then take all steps 
>> necessary to permanently delete the email and all attachments from your 
>> computer system.
>> 
> 

Re: CGNAT

[Kevin Burke]

Hi Steve

We are looking at implementing a similar solution with A10 for CGNAT.

We've been in touch with A10. Just wondering if there are some alternative 
vendors that anyone would recommend. We'd probably be looking at a solution to 
support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting 
point. A solution that is as transparent to user experience as possible is a 
priority.


The numbers below are for a similar target of subscriber’s and peak bandwidth.

We assumed a couple of numbers:
Current Peak Bandwidth = 40G
Remaining IPv4 traffic after migration = 20% (Seen references to 10% or 20% on 
this forum)
Future Bandwidth Growth = 2x (no data behind this assumption)
Future CGNAT’ed bandwidth = 15Gbps
Equipment & budget lifecycle = 7Yr

Getting that data led us to this price comparison:

Solution
Lifecycle/ Term
Annual Cost/Sub
Product Lifecycle Cost/Sub
Lease IPv4 Cogent
7
$ 4.45
 $   31.13
A10 CGNAT 15Gb 7Yr
7
$ 1.21
 $ 8.47
A10 CGNAT 40Gb 7Yr
7
$ 1.95
 $   13.68
Purchase @ $25 7Yr
7
$ 3.57
 $   25.00


The current plan is implement an A10 CGNAT solution after upgrading our network 
for IPv6.  In the interim we will have to lease IPv4 to tide us over.

I would be curious to see what other’s estimate the costs of various 
approaches.  Feel free to ping me off-list for more specific numbers.

Kevin Burke
802-540-0979
Burlington Telecom
200 Church St, Burlington, VT

From: NANOG  on behalf of 
Steve Saner 
Date: Friday, February 19, 2021 at 9:56 AM
To: "nanog@nanog.org" 
Subject: CGNAT

We are starting to look at CGNAT solutions. The primary motivation at the 
moment is to extend current IPv4 resources, but IPv6 migration is also a factor.

We've been in touch with A10. Just wondering if there are some alternative 
vendors that anyone would recommend. We'd probably be looking at a solution to 
support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting 
point. A solution that is as transparent to user experience as possible is a 
priority.

Thanks

--
Steve Saner
ideatek HUMAN AT OUR VERY FIBER

This email transmission, and any documents, files or previous email messages 
attached to it may contain confidential information. If the reader of this 
message is not the intended recipient or the employee or agent responsible for 
delivering the message to the intended recipient, you are hereby notified that 
any dissemination, distribution or copying of this communication is strictly 
prohibited. If you are not, or believe you may not be, the intended recipient, 
please advise the sender immediately by return email or by calling 
620.543.5026. Then take all steps necessary to permanently 
delete the email and all attachments from your computer system.

RE: CGNAT

[na...@jima.us]

While I don't doubt the accuracy of Lee's presentation at the time, at least 
two base factors have changed since then:

- Greater deployment of IPv6 content (necessitating less CGN capacity per user)
- Increased price of Legacy IP space on the secondary market (changing the 
formula) -- strictly speaking, this presentation was still in "primary market" 
era for LACNIC/ARIN/AFRINIC

IPv6 migration is not generally aided by CGNAT, but CGNAT deployment is 
generally aided by IPv6 deployment; to reiterate the earlier point, any ISPs 
deploying CGNAT without first deploying IPv6 are burning cash.

- Jima

From: NANOG On Behalf Of Owen DeLong
Sent: Sunday, February 21, 2021 16:59
To: Steve Saner
Cc: nanog@nanog.org
Subject: Re: CGNAT


On Feb 18, 2021, at 8:38 AM, Steve Saner wrote:

> We are starting to look at CGNAT solutions. The primary motivation at the 
> moment is to extend current IPv4 resources, but IPv6 migration is also a 
> factor.

IPv6 Migration is generally not aided by CGNAT.

In general, the economics today still work out to make purchasing or leasing 
addresses more favorable than CGNAT.

It’s a bit dated by now, but still very relevant, see Lee Howard’s excellent 
research presented at the 2012 Rocky
mountain v6 task force meeting:

https://www.rmv6tf.org/wp-content/uploads/2012/11/TCO-of-CGN1.pdf

Owen


We've been in touch with A10. Just wondering if there are some alternative 
vendors that anyone would recommend. We'd probably be looking at a solution to 
support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting 
point. A solution that is as transparent to user experience as possible is a 
priority.

Thanks

-- 
Steve Saner
ideatek HUMAN AT OUR VERY FIBER
This email transmission, and any documents, files or previous email messages 
attached to it may contain confidential information. If the reader of this 
message is not the intended recipient or the employee or agent responsible for 
delivering the message to the intended recipient, you are hereby notified that 
any dissemination, distribution or copying of this communication is strictly 
prohibited. If you are not, or believe you may not be, the intended recipient, 
please advise the sender immediately by return email or by calling 
tel:620.543.5026. Then take all steps necessary to permanently delete the email 
and all attachments from your computer system.

Re: CGNAT

[Owen DeLong]

> On Feb 18, 2021, at 8:38 AM, Steve Saner  wrote:
> 
> We are starting to look at CGNAT solutions. The primary motivation at the 
> moment is to extend current IPv4 resources, but IPv6 migration is also a 
> factor.

IPv6 Migration is generally not aided by CGNAT.

In general, the economics today still work out to make purchasing or leasing 
addresses more favorable than CGNAT.

It’s a bit dated by now, but still very relevant, see Lee Howard’s excellent 
research presented at the 2012 Rocky
mountain v6 task force meeting:

https://www.rmv6tf.org/wp-content/uploads/2012/11/TCO-of-CGN1.pdf 


Owen

> We've been in touch with A10. Just wondering if there are some alternative 
> vendors that anyone would recommend. We'd probably be looking at a solution 
> to support 5k to 15k customers and bandwidth up to around 30-40 gig as a 
> starting point. A solution that is as transparent to user experience as 
> possible is a priority.
> 
> Thanks
> 
> -- 
> Steve Saner
> ideatek HUMAN AT OUR VERY FIBER
> This email transmission, and any documents, files or previous email messages 
> attached to it may contain confidential information. If the reader of this 
> message is not the intended recipient or the employee or agent responsible 
> for delivering the message to the intended recipient, you are hereby notified 
> that any dissemination, distribution or copying of this communication is 
> strictly prohibited. If you are not, or believe you may not be, the intended 
> recipient, please advise the sender immediately by return email or by calling 
> 620.543.5026 . Then take all steps necessary to permanently 
> delete the email and all attachments from your computer system.
> 

Re: CGNAT

[Tom Hill]

On 19/02/2021 20:11, Tony Wicks wrote:
> Because then a large part of the Internet won't work

Hey, look on the bright side: customers won't be able to use Twitter to
complain! :D

Ofc, IPv4aaS has many good success stories out there; Sky Italia are
running MAP-T, many, many mobile ISPs are running 464XLAT with great
success.

We're in a situation where making IPv6 a *prerequisite* of your IPv4
connectivity can realistically improve your margins when some sort of
CGNAT gateway is a requirement.

Yes it requires looking at your CPE support, but if you're doing even
00,000's of subs, I'm sure the benefits aren't trivial when viewed
through the lens of the number of connections that a single Chrome tab
can happily chew through.

-- 
Tom

Re: CGNAT

[Mark Andrews]

I’m sure the large parts of the world already doing this would disagree.
-- 
Mark Andrews

> On 20 Feb 2021, at 07:11, Tony Wicks  wrote:
> 
> 
> Because then a large part of the Internet won't work
> 
> From: NANOG  on behalf of Mark 
> Andrews 
> Sent: Saturday, 20 February 2021, 9:04 am
> To: Steve Saner
> Cc: nanog@nanog.org
> Subject: Re: CGNAT
> 
> Why not go whole hog and provide IPv4 as a service? That way you are not 
> waiting for your customers to turn up IPv6 to take the load off your NAT box.
> 
> Yes, you can do it dual stack but you have waited so long you may as well 
> miss that step along the deployment path.
> -- 
> Mark Andrews
> 
>>> On 20 Feb 2021, at 01:55, Steve Saner  wrote:
>>> 
>> 
>> We are starting to look at CGNAT solutions. The primary motivation at the 
>> moment is to extend current IPv4 resources, but IPv6 migration is also a 
>> factor.
>> 
>> We've been in touch with A10. Just wondering if there are some alternative 
>> vendors that anyone would recommend. We'd probably be looking at a solution 
>> to support 5k to 15k customers and bandwidth up to around 30-40 gig as a 
>> starting point. A solution that is as transparent to user experience as 
>> possible is a priority.
>> 
>> Thanks
>> 
>> -- 
>> Steve Saner
>> ideatek HUMAN AT OUR VERY FIBER
>> This email transmission, and any documents, files or previous email messages 
>> attached to it may contain confidential information. If the reader of this 
>> message is not the intended recipient or the employee or agent responsible 
>> for delivering the message to the intended recipient, you are hereby 
>> notified that any dissemination, distribution or copying of this 
>> communication is strictly prohibited. If you are not, or believe you may not 
>> be, the intended recipient, please advise the sender immediately by return 
>> email or by calling 620.543.5026. Then take all steps necessary to 
>> permanently delete the email and all attachments from your computer system.
>> 
> 

Re: CGNAT

[JORDI PALET MARTINEZ via NANOG]

IPv4 as a Service such as 464XLAT, will allow them to use less IPv4 public 
addresses than CGNAT, less costly equipment (or open source) and still provide 
dual-stack inside the customers networks.

 

There is nothing from Internet that will not work. I’ve many deployments based 
on this, and this is the transition mechanism that have more millions of 
customers, even if compared with all the others together.

 

Regards,

Jordi

@jordipalet

 

 

 

El 19/2/21 21:15, "NANOG en nombre de Tony Wicks" 
 escribió:

 

Because then a large part of the Internet won't work

 

From: NANOG  on behalf of Mark 
Andrews 
Sent: Saturday, 20 February 2021, 9:04 am
To: Steve Saner
Cc: nanog@nanog.org
Subject: Re: CGNAT


Why not go whole hog and provide IPv4 as a service? That way you are not 
waiting for your customers to turn up IPv6 to take the load off your NAT box.

 

Yes, you can do it dual stack but you have waited so long you may as well miss 
that step along the deployment path.

-- 

Mark Andrews




On 20 Feb 2021, at 01:55, Steve Saner  wrote:



We are starting to look at CGNAT solutions. The primary motivation at the 
moment is to extend current IPv4 resources, but IPv6 migration is also a factor.

 

We've been in touch with A10. Just wondering if there are some alternative 
vendors that anyone would recommend. We'd probably be looking at a solution to 
support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting 
point. A solution that is as transparent to user experience as possible is a 
priority.

 

Thanks


-- 

Steve Saner

ideatek HUMAN AT OUR VERY FIBER

This email transmission, and any documents, files or previous email messages 
attached to it may contain confidential information. If the reader of this 
message is not the intended recipient or the employee or agent responsible for 
delivering the message to the intended recipient, you are hereby notified that 
any dissemination, distribution or copying of this communication is strictly 
prohibited. If you are not, or believe you may not be, the intended recipient, 
please advise the sender immediately by return email or by calling 
620.543.5026. Then take all steps necessary to permanently delete the email and 
all attachments from your computer system.

 



**
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or 
confidential. The information is intended to be for the exclusive use of the 
individual(s) named above and further non-explicilty authorized disclosure, 
copying, distribution or use of the contents of this information, even if 
partially, including attached files, is strictly prohibited and will be 
considered a criminal offense. If you are not the intended recipient be aware 
that any disclosure, copying, distribution or use of the contents of this 
information, even if partially, including attached files, is strictly 
prohibited, will be considered a criminal offense, so you must reply to the 
original sender to inform about this communication and delete it.

Re: CGNAT

[Tony Wicks]

Because then a large part of the Internet won't workFrom: NANOG  on behalf of Mark Andrews Sent: Saturday, 20 February 2021, 9:04 amTo: Steve SanerCc: nanog@nanog.orgSubject: Re: CGNATWhy not go whole hog and provide IPv4 as a service? That way you are not waiting for your customers to turn up IPv6 to take the load off your NAT box.Yes, you can do it dual stack but you have waited so long you may as well miss that step along the deployment path.-- Mark AndrewsOn 20 Feb 2021, at 01:55, Steve Saner  wrote:We are starting to look at CGNAT solutions. The primary motivation at the moment is to extend current IPv4 resources, but IPv6 migration is also a factor.We've been in touch with A10. Just wondering if there are some alternative vendors that anyone would recommend. We'd probably be looking at a solution to support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting point. A solution that is as transparent to user experience as possible is a priority.Thanks-- Steve Sanerideatek HUMAN AT OUR VERY FIBERThis
 email transmission, and any documents, files or previous email messages
 attached to it may contain confidential information. If the reader of 
this message is not the intended recipient or the employee or agent 
responsible for delivering the message to the intended recipient, you 
are hereby notified that any dissemination, distribution or copying of 
this communication is strictly prohibited. If you are not, or believe 
you may not be, the intended recipient, please advise the sender 
immediately by return email or by calling 620.543.5026. Then take all steps necessary to permanently delete the email and all attachments from your computer system.

Re: CGNAT

[Mark Andrews]

Why not go whole hog and provide IPv4 as a service? That way you are not 
waiting for your customers to turn up IPv6 to take the load off your NAT box.

Yes, you can do it dual stack but you have waited so long you may as well miss 
that step along the deployment path.
-- 
Mark Andrews

> On 20 Feb 2021, at 01:55, Steve Saner  wrote:
> 
> 
> We are starting to look at CGNAT solutions. The primary motivation at the 
> moment is to extend current IPv4 resources, but IPv6 migration is also a 
> factor.
> 
> We've been in touch with A10. Just wondering if there are some alternative 
> vendors that anyone would recommend. We'd probably be looking at a solution 
> to support 5k to 15k customers and bandwidth up to around 30-40 gig as a 
> starting point. A solution that is as transparent to user experience as 
> possible is a priority.
> 
> Thanks
> 
> -- 
> Steve Saner
> ideatek HUMAN AT OUR VERY FIBER
> This email transmission, and any documents, files or previous email messages 
> attached to it may contain confidential information. If the reader of this 
> message is not the intended recipient or the employee or agent responsible 
> for delivering the message to the intended recipient, you are hereby notified 
> that any dissemination, distribution or copying of this communication is 
> strictly prohibited. If you are not, or believe you may not be, the intended 
> recipient, please advise the sender immediately by return email or by calling 
> 620.543.5026. Then take all steps necessary to permanently delete the email 
> and all attachments from your computer system.

RE: CGNAT

[Tony Wicks]

Not the Cheapest option out there but the most rock solid one I have found is 
to install the extended service/multi service cards in the BNG and do it 
locally there. We are currently using both Juniper MX480/960 with MS-MPC cards 
and Nokia 7750 SR with ISA or ESA cards. Its also well worth running dual stack 
IPv6 as you can bypass 40%+ traffic from the CGN process for all that CDN 
traffic.

 

From: NANOG  On Behalf Of Steve Saner
Sent: Friday, 19 February 2021 5:39 am
To: nanog@nanog.org
Subject: CGNAT

 

We are starting to look at CGNAT solutions. The primary motivation at the 
moment is to extend current IPv4 resources, but IPv6 migration is also a factor.

 

We've been in touch with A10. Just wondering if there are some alternative 
vendors that anyone would recommend. We'd probably be looking at a solution to 
support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting 
point. A solution that is as transparent to user experience as possible is a 
priority.

 

Thanks


-- 

Steve Saner

ideatek HUMAN AT OUR VERY FIBER

This email transmission, and any documents, files or previous email messages 
attached to it may contain confidential information. If the reader of this 
message is not the intended recipient or the employee or agent responsible for 
delivering the message to the intended recipient, you are hereby notified that 
any dissemination, distribution or copying of this communication is strictly 
prohibited. If you are not, or believe you may not be, the intended recipient, 
please advise the sender immediately by return email or by calling  
 620.543.5026. Then take all steps necessary to permanently 
delete the email and all attachments from your computer system.

Re: CGNAT

[Douglas Fischer]

I recommend you to take a look at DANOS.

https://danosproject.atlassian.net/wiki/spaces/DAN/pages/416153601/Carrier+Grade+NAT+CGNAT

- A very active open-source project.
- Sponsored by AT
- Uses Vyatta (and DPDK for good performance)
- The Routing Engine is based on FRR.
- Syntax sounds like Junos.
- Is the ONLY ONE open source project(at least that I know) that implements
CGNAT on Bulk Port Allocation mode(not deterministic/predefined).
- Had very good improvements on PCP recently.
- Supports a few NAT-ALGs.

I and some good friends here in Brazil had some good experiences with it.

Marcelo Gondin wrote this tutorial in pt_BR, mentioning about a case with:
26Gbps / 1.5Mpps / 11502 simultaneous clients / 192 used Públic IPv4
addresses.
https://wiki.brasilpeeringforum.org/w/CGNAT_Bulk_Port_Allocation_com_DPDK



Em sex., 19 de fev. de 2021 às 11:57, Steve Saner 
escreveu:

> We are starting to look at CGNAT solutions. The primary motivation at the
> moment is to extend current IPv4 resources, but IPv6 migration is also a
> factor.
>
> We've been in touch with A10. Just wondering if there are some alternative
> vendors that anyone would recommend. We'd probably be looking at a solution
> to support 5k to 15k customers and bandwidth up to around 30-40 gig as a
> starting point. A solution that is as transparent to user experience as
> possible is a priority.
>
> Thanks
>
> --
> Steve Saner
> ideatek HUMAN AT OUR VERY FIBER
>
> This email transmission, and any documents, files or previous email
> messages attached to it may contain confidential information. If the reader
> of this message is not the intended recipient or the employee or agent
> responsible for delivering the message to the intended recipient, you are
> hereby notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not, or believe you may
> not be, the intended recipient, please advise the sender immediately by
> return email or by calling 620.543.5026. Then take all steps necessary to
> permanently delete the email and all attachments from your computer system.
>


-- 
Douglas Fernando Fischer
Engº de Controle e Automação

Re: CGNAT Opensource with support to BPA, EIM/EIF, UPnP-PCP

[Mark Tinka]

On 7/Jul/20 19:23, JORDI PALET MARTINEZ via NANOG wrote:

>  
>
> There was, long time ago, something developed by ISC, but I think
> never completed and not updated …
>
>  
>
> 464XLAT is always a solution and becomes much cheaper, than CGN from
> vendors, even if you need to replace the CPEs. I’m doing that now with
> 25.000.000 subscribers … (slowed down by the Covid-19).
>

I have to agree... as "transition" tech. goes, 464XLAT is the least
intrusive solution, because as more of your customers acquire IPv6, the
demands you put on your 464XLAT systems reduce, naturally. It also means
you don't have to carry out yet-another transition to get the full IPv6
experience.

Mark.

Re: CGNAT Opensource with support to BPA, EIM/EIF, UPnP-PCP

[Mark Andrews]

> On 8 Jul 2020, at 03:23, JORDI PALET MARTINEZ via NANOG  
> wrote:
> 
> Hi Douglas,
>  
> There was, long time ago, something developed by ISC, but I think never 
> completed and not updated …

ISC did a DS-LITE implementation called AFTR.  This can be found at:

https://ftp.isc.org/isc/aftr/

> 464XLAT is always a solution and becomes much cheaper, than CGN from vendors, 
> even if you need to replace the CPEs. I’m doing that now with 25.000.000 
> subscribers … (slowed down by the Covid-19).
>  
> Regards,
> Jordi
> 
> @jordipalet
> 
>  
> 
>  
>  
> El 7/7/20 18:44, "NANOG en nombre de Douglas Fischer" 
>  fischerdoug...@gmail.com> escribió:
>  
> We are looking for a CGNAT solution open source based.
> 
> Yep, I know that basic CGNAT can be done with iptables / nftables, or PF / 
> IPFILTER / IPFW.
> 
> But I only know Open Source CGNAT recipes with predefined public-ports <-> 
> private IPs mapping.
> 
> What It brings two types of issues:
> A - The need to overprovision the number of private IPs (Considering Multiple 
> BNGs behind the CGN).
> B - The inability of those basic recipes to deal with incoming auxiliary 
> connections of p2p protocols (mostly used by games).
> 
> Te market solutions that I've dealt with solves those issues beautifully.
> a - Bulk-Port Allocation - BPA, avoid the need overprovisioning private 
> address that is not being used, and give us an excellent rate between public 
> IPv4 Address vs Private IP Address.
> b - The support of a framework of protocols(Ex.: UPnP, PCP, EIM/EIF, NAT-PMP, 
> etc...) ensure an acceptable quality of experience to end-users.
> 
> But, the market solution brings also some down-sides...
> - The cost, evidently.
> - The need for detouring the traffic that doesn't need CGNAT(Internal CDNs, 
> Internal Servers, etc), to stay on the license limits of those boxes, 
> sometimes brings some issues.
> 
> So, I and some friends are(for a long time) looking for an OpenSource 
> solution that can give us something near what the market solutions give.
> 
> Any of you guys ave some suggestions for that?
> 
> 
> P.S.: Yes, I know that IPv6 is the only real solution for that, but until 
> there, our customers still want to access a lot os p2p content(mostly audio 
> in game rooms, sip calls, and things like that.)
> 
> P.S.2: Yes, I also know that 464 could be a good possibility, but is not 
> possible in this scenario.
>  
> -- 
> Douglas Fernando Fischer
> Engº de Controle e Automação
> 
> **
> IPv4 is over
> Are you ready for the new Internet ?
> http://www.theipv6company.com
> The IPv6 Company
> 
> This electronic message contains information which may be privileged or 
> confidential. The information is intended to be for the exclusive use of the 
> individual(s) named above and further non-explicilty authorized disclosure, 
> copying, distribution or use of the contents of this information, even if 
> partially, including attached files, is strictly prohibited and will be 
> considered a criminal offense. If you are not the intended recipient be aware 
> that any disclosure, copying, distribution or use of the contents of this 
> information, even if partially, including attached files, is strictly 
> prohibited, will be considered a criminal offense, so you must reply to the 
> original sender to inform about this communication and delete it.

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742  INTERNET: ma...@isc.org

RE: CGNAT Opensource with support to BPA, EIM/EIF, UPnP-PCP

[Tony Wicks]

As someone who has spent quite a long time building CGNAT solutions I have some 
good news for you, there is an easy solution to your below point that works 
exceptionally well. The solution is dual stack IPv6, its trivial to route your 
IPv6 to bypass the CGNAT device you are using and pretty much all of the major 
CDN providers are fully IPv6 enabled. In the real world this halves the amount 
of traffic your CGNAT solution has to process. Gaming companies (Not Sony) 
are also starting to support V6 so that can be a win too. I’m not one of those 
V6 is the solution to everything engineers as I live in the real world, but in 
this case it absolutely is a good workable answer.

 


- The need for detouring the traffic that doesn't need CGNAT(Internal CDNs, 
Internal Servers, etc), to stay on the license limits of those boxes, sometimes 
brings some issues.

Re: CGNAT Opensource with support to BPA, EIM/EIF, UPnP-PCP

[Jared Geiger]

DANOS 2005 seems to support a lot of your requirements.
https://danosproject.atlassian.net/wiki/spaces/DAN/pages/320634926/DANOS+2005+Release+Notes

So if you have an x86 box with supported NICS you should be able to get
some decent performance from it.

The major gotcha in this release is I think route-maps, prefix-lists,
access-lists with BGP are broken.

On Tue, Jul 7, 2020 at 9:44 AM Douglas Fischer 
wrote:

> We are looking for a CGNAT solution open source based.
>
> Yep, I know that basic CGNAT can be done with iptables / nftables, or PF /
> IPFILTER / IPFW.
>
> But I only know Open Source CGNAT recipes with predefined public-ports <->
> private IPs mapping.
>
> What It brings two types of issues:
> A - The need to overprovision the number of private IPs (Considering
> Multiple BNGs behind the CGN).
> B - The inability of those basic recipes to deal with incoming auxiliary
> connections of p2p protocols (mostly used by games).
>
> Te market solutions that I've dealt with solves those issues beautifully.
> a - Bulk-Port Allocation - BPA, avoid the need overprovisioning private
> address that is not being used, and give us an excellent rate between
> public IPv4 Address vs Private IP Address.
> b - The support of a framework of protocols(Ex.: UPnP, PCP, EIM/EIF,
> NAT-PMP, etc...) ensure an acceptable quality of experience to end-users.
>
> But, the market solution brings also some down-sides...
> - The cost, evidently.
> - The need for detouring the traffic that doesn't need CGNAT(Internal
> CDNs, Internal Servers, etc), to stay on the license limits of those boxes,
> sometimes brings some issues.
>
> So, I and some friends are(for a long time) looking for an OpenSource
> solution that can give us something near what the market solutions give.
>
> Any of you guys ave some suggestions for that?
>
>
> P.S.: Yes, I know that IPv6 is the only real solution for that, but until
> there, our customers still want to access a lot os p2p content(mostly audio
> in game rooms, sip calls, and things like that.)
>
> P.S.2: Yes, I also know that 464 could be a good possibility, but is not
> possible in this scenario.
>
> --
> Douglas Fernando Fischer
> Engº de Controle e Automação
>

Re: CGNAT Opensource with support to BPA, EIM/EIF, UPnP-PCP

[JORDI PALET MARTINEZ via NANOG]

Hi Douglas,

 

There was, long time ago, something developed by ISC, but I think never 
completed and not updated …

 

464XLAT is always a solution and becomes much cheaper, than CGN from vendors, 
even if you need to replace the CPEs. I’m doing that now with 25.000.000 
subscribers … (slowed down by the Covid-19).

 

Regards,

Jordi

@jordipalet

 

 

 

El 7/7/20 18:44, "NANOG en nombre de Douglas Fischer" 
 escribió:

 

We are looking for a CGNAT solution open source based.

Yep, I know that basic CGNAT can be done with iptables / nftables, or PF / 
IPFILTER / IPFW.

But I only know Open Source CGNAT recipes with predefined public-ports <-> 
private IPs mapping.

What It brings two types of issues:
A - The need to overprovision the number of private IPs (Considering Multiple 
BNGs behind the CGN).
B - The inability of those basic recipes to deal with incoming auxiliary 
connections of p2p protocols (mostly used by games).

Te market solutions that I've dealt with solves those issues beautifully.
a - Bulk-Port Allocation - BPA, avoid the need overprovisioning private address 
that is not being used, and give us an excellent rate between public IPv4 
Address vs Private IP Address.
b - The support of a framework of protocols(Ex.: UPnP, PCP, EIM/EIF, NAT-PMP, 
etc...) ensure an acceptable quality of experience to end-users.

But, the market solution brings also some down-sides...
- The cost, evidently.
- The need for detouring the traffic that doesn't need CGNAT(Internal CDNs, 
Internal Servers, etc), to stay on the license limits of those boxes, sometimes 
brings some issues.

So, I and some friends are(for a long time) looking for an OpenSource solution 
that can give us something near what the market solutions give.

Any of you guys ave some suggestions for that?


P.S.: Yes, I know that IPv6 is the only real solution for that, but until 
there, our customers still want to access a lot os p2p content(mostly audio in 
game rooms, sip calls, and things like that.)

P.S.2: Yes, I also know that 464 could be a good possibility, but is not 
possible in this scenario.

 

-- 

Douglas Fernando Fischer
Engº de Controle e Automação



**
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or 
confidential. The information is intended to be for the exclusive use of the 
individual(s) named above and further non-explicilty authorized disclosure, 
copying, distribution or use of the contents of this information, even if 
partially, including attached files, is strictly prohibited and will be 
considered a criminal offense. If you are not the intended recipient be aware 
that any disclosure, copying, distribution or use of the contents of this 
information, even if partially, including attached files, is strictly 
prohibited, will be considered a criminal offense, so you must reply to the 
original sender to inform about this communication and delete it.

Re: CGNAT Solutions

[Masataka Ohta]

Ca By wrote:


The proper number to be considered should be percentage of IPv6
hosts which can not communicate with IPv4 only hosts.

Isn't it 0%?


I think you agree with me, here.


For those of us running networks, especially growing networks, uniquely
numbering hosts is our goal and ipv6 fits that task.


Then, you should be running some isolated network.

In this thread, we, except you, are discussing how to uniquely identify
customers, not hosts, without (much) logging.


For many networks, rfc1918 space is not sufficiently large to number
end-points. Around the world, there are many networks that fit this.


The global address space of IPv4 with NAT is combination of IPv4
address and part of port number spaces, which should be enough to
identify customers and, maybe, hosts. and is much larger than
private space of rfc1918.

> So far, i just talked about why eyeball networks deploy ipv6 — which is
> basic and sensible engineering and economics.  A similar set of 
forces are

> at work on the content / cloud / iot side.

Perfect argument for OSI.

Masataka Ohta

Re: CGNAT Solutions

[JORDI PALET MARTINEZ via NANOG]

And more and more CPE providers support it.

 

See RFC8585.

 

I inititally started using OpenWRT, but now I already got samples from several 
vendors.

 

Regards,

Jordi

@jordipalet

 

 

 

El 30/4/20 6:16, "NANOG en nombre de Ca By"  escribió:

 

 

 

On Wed, Apr 29, 2020 at 7:17 PM Brandon Martin  wrote:

On 4/29/20 10:12 PM, William Herrin wrote:
>> What allows them to work with v6 in such an efficient manner?
> A piece of client software is installed on every phone that presents
> an IPv4 address to the phone and then translates packets to IPv6 for
> relay over the network. This works because T-Mobile has considerable
> control over the phone.

FWIW, this software component (the CLAT) can also be on the CPE edge 
router which many ISPs either control outright these days or at least 
can influence.
-- 
Brandon Martin

 

Correct, and T-Mobile uses this 464xlat approach for their home broadband 
product as well

 

 



**
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or 
confidential. The information is intended to be for the exclusive use of the 
individual(s) named above and further non-explicilty authorized disclosure, 
copying, distribution or use of the contents of this information, even if 
partially, including attached files, is strictly prohibited and will be 
considered a criminal offense. If you are not the intended recipient be aware 
that any disclosure, copying, distribution or use of the contents of this 
information, even if partially, including attached files, is strictly 
prohibited, will be considered a criminal offense, so you must reply to the 
original sender to inform about this communication and delete it.

Re: CGNAT Solutions

[Ca By]

On Wed, Apr 29, 2020 at 7:17 PM Brandon Martin 
wrote:

> On 4/29/20 10:12 PM, William Herrin wrote:
> >> What allows them to work with v6 in such an efficient manner?
> > A piece of client software is installed on every phone that presents
> > an IPv4 address to the phone and then translates packets to IPv6 for
> > relay over the network. This works because T-Mobile has considerable
> > control over the phone.
>
> FWIW, this software component (the CLAT) can also be on the CPE edge
> router which many ISPs either control outright these days or at least
> can influence.
> --
> Brandon Martin


Correct, and T-Mobile uses this 464xlat approach for their home broadband
product as well


>

Re: CGNAT Solutions

[Ca By]

On Wed, Apr 29, 2020 at 7:46 PM Masataka Ohta <
mo...@necom830.hpcl.titech.ac.jp> wrote:

> Ca By wrote:
>
> >>>You can't eliminate that unless the CPE also knows what internal
> port
> >>> range it's mapped to so that it restricts what range it uses.  If you
> >>> can do that, you can get rid of the programmatic state tracking
> entirely
> >>> and just use static translations for TCP and UDP which, while nice, is
> >>> impractical.  You're about 95% of the way to LW4o6 or MAP at that
> point.
> >>
> >> Interesting. Then, if you can LW4o6 or MAP, you are about 95% of the
> >> way to E2ENAT with complete end to end transparency using IPv4 only,
> >> which means we don't need IPv6 with 4to6 NAT lacking the transparency.
> >>
> >>  https://tools.ietf.org/html/draft-ohta-e2e-nat-00
> >>
> >>  Masataka Ohta
>
> > Since we are talking numbers ans hard facts
>
> I'm rather interested in not numbers but facts on the E2E
> transparency, because, without the transparency, legacy
> NAT44 should be enough.
>
> But, as you insist on numbers:
>
> > 42% of usa accesses google on ipv6
> >
> > https://www.google.com/intl/en/ipv6/statistics.html
>
> The proper number to be considered should be percentage of IPv6
> hosts which can not communicate with IPv4 only hosts.
>
> Isn't it 0%?


For those of us running networks, especially growing networks, uniquely
numbering hosts is our goal and ipv6 fits that task.

For many networks, rfc1918 space is not sufficiently large to number
end-points. Around the world, there are many networks that fit this.

For those same network, nat44 scale is also a painful and costly effort.

To that end, ipv6 / 464xlat provides the one-two punch of uniquely
numbering nodes and by-passing NAT44 or NAT64 for the majority of traffic
we see (google, fb, netflix ...)

Being able to offer a product that disallows access to ipv4 is a non-goal

So far, i just talked about why eyeball networks deploy ipv6 — which is
basic and sensible engineering and economics.  A similar set of forces are
at work on the content / cloud / iot side.



>
> Masataka Ohta
>

Re: CGNAT Solutions

[Masataka Ohta]

Ca By wrote:


   You can't eliminate that unless the CPE also knows what internal port
range it's mapped to so that it restricts what range it uses.  If you
can do that, you can get rid of the programmatic state tracking entirely
and just use static translations for TCP and UDP which, while nice, is
impractical.  You're about 95% of the way to LW4o6 or MAP at that point.


Interesting. Then, if you can LW4o6 or MAP, you are about 95% of the
way to E2ENAT with complete end to end transparency using IPv4 only,
which means we don't need IPv6 with 4to6 NAT lacking the transparency.

 https://tools.ietf.org/html/draft-ohta-e2e-nat-00

 Masataka Ohta



Since we are talking numbers ans hard facts


I'm rather interested in not numbers but facts on the E2E
transparency, because, without the transparency, legacy
NAT44 should be enough.

But, as you insist on numbers:


42% of usa accesses google on ipv6

https://www.google.com/intl/en/ipv6/statistics.html


The proper number to be considered should be percentage of IPv6
hosts which can not communicate with IPv4 only hosts.

Isn't it 0%?

Masataka Ohta

Re: CGNAT Solutions

[Brandon Martin]

On 4/29/20 10:12 PM, William Herrin wrote:

What allows them to work with v6 in such an efficient manner?

A piece of client software is installed on every phone that presents
an IPv4 address to the phone and then translates packets to IPv6 for
relay over the network. This works because T-Mobile has considerable
control over the phone.


FWIW, this software component (the CLAT) can also be on the CPE edge 
router which many ISPs either control outright these days or at least 
can influence.

--
Brandon Martin

Re: CGNAT Solutions

[William Herrin]

On Wed, Apr 29, 2020 at 5:27 PM Thomas Scott  wrote:
> > cell-phone environment. A classic small ISP fills a different niche.
>
> I've dealt with traditional cable and fiber SP environments, but I'm curious 
> how the architecture differs so drastically with T-Mobile to allow v6 to work 
> so seamlessly. From working with their techs when turning up circuits, I 
> don't feel like their architecture and the ones I've worked in differ too 
> dramatically. My private cell service is through a tmo reseller, and I've 
> only ever had a public v6 address on them.
>
> What allows them to work with v6 in such an efficient manner?

A piece of client software is installed on every phone that presents
an IPv4 address to the phone and then translates packets to IPv6 for
relay over the network. This works because T-Mobile has considerable
control over the phone.

https://www.internetsociety.org/resources/deploy360/2014/case-study-t-mobile-us-goes-ipv6-only-using-464xlat/

Regards,
Bill Herrin


-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: CGNAT Solutions

[William Herrin]

On Wed, Apr 29, 2020 at 7:19 AM Ca By  wrote:
> Since we are talking numbers ans hard facts
>
> 42% of usa accesses google on ipv6
>
> https://www.google.com/intl/en/ipv6/statistics.html

Be careful with those stats; they might not be telling you what you
think they are. For example, phone clients are characteristically
different than classic ISP clients. A substantial portion, perhaps
majority of Google's use comes from phone clients but the numbers
aren't broken out in those roll-up stats.

Among others, T-Mobile has demonstrated that a v6-only infrastructure
(with some v4 and NAT at the border) is credible and achievable in a
cell-phone environment. A classic small ISP fills a different niche.

Regards,
Bill Herrin

-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

RE: CGNAT Solutions

[Aaron Gould]

In testing, I observed opening a website, for instance cnn.com can cause >200 
ports/sessions to fire off.  Although, many are short-lived sessions, but, 
ports requests nonetheless.

Overall, I use about 1,500 public ip's for 50,000 private ip customers

I allow 3,000 ports per customer ... 30 blocks of 100 each

We started our port blocks at a nice round number, so that each pba dynamic 
assignment results in nice 100-199, next 200-299  good for parsing, 
grep'ing logs for doing subpoena info look-ups, etc.

I see most customers hover well below 1,000 ports/sessions active, and what 
appear to be misbehaving hosts (malware, infected, bots, etc, unsure) hit up at 
the 3,000 max and trigger a ports exceeded error message.  I see the 3k port 
limit as putting a cap on free-running suspicious hosts.  We can then 
investigate and contact customer of the concern.

-Aaron


-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Robert Blayzor
Sent: Wednesday, April 29, 2020 9:14 AM
To: nanog@nanog.org
Subject: Re: CGNAT Solutions

On 4/28/20 11:01 PM, Brandon Martin wrote:
> Depending on how many IPs you need to reclaim and what your target
> IP:subscriber ratio is, you may be able to eliminate the need for a lot
> of logging by assigning a range of TCP/UDP ports to a single inside IP
> so that the TCP/UDP port number implies a specific subscriber.
> 
> You can't get rid of all the state tracking without also having the CPE
> know which ports to use (in which case you might as well use LW4o6 or
> MAP), but at least you can get it down to where you really only need to
> log (or block and dole out public IPs as needed) port-less protocols.


I'm wondering if there are any real world examples of this, namely in
the realm of subscriber to IP and range of ports required, etc.  ie: Is
is a range of 1000 ports enough for one residential subscriber? How
about SMB where no global IP is required.

One would think a 1000 ports would be enough, but if you have a dozen
devices at home all browsing and doing various things, and with IOT,
etc, maybe not?


-- 
inoc.net!rblayzor
XMPP: rblayzor.AT.inoc.net
PGP:  https://pgp.inoc.net/rblayzor/

Re: CGNAT Solutions

[Mikael Abrahamsson via NANOG]

On Wed, 29 Apr 2020, Robert Blayzor wrote:

So as a happy medium of about 2048 ports per subscriber, that's roughly 
a 32:1 NAT/IP over-subscription ?


Yes, around that.

--
Mikael Abrahamssonemail: swm...@swm.pp.se

Re: CGNAT Solutions

[John Alcock]

Thank you everyone for the suggestions.

To clarify small ISP.

12K subscribers
35 Gigs traffic at peak.

Growing about 500 megs per month traffic.

John

On Tue, Apr 28, 2020 at 3:12 PM John Alcock  wrote:

> Afternoon,
>
> I run a small ISP in Tennessee.  COVID has forced a lot of people to work
> from home.  I am starting to run low on IP's and need to consider CGNAT.
>
> I do have IPV6 space, but we all know that until we force everyone to move
> to IPV6, we need to keep IPV4 up and running.
>
> I could buy more space, but I am really wondering if that is the
> best option.  It is expensive. I know CGNAT devices are expensive as well,
> but it looks like I could stretch it out a bit.
>
> My thinking is to convert about 50% of my subscribers to CGNAT.
>
> I am interested in vendors or devices you have used in the past.  I
> already know about the pitfalls many of my subscribers will have with CGNAT
> such as VPN's, Gamers, etc.
>
> What are your thoughts on CGNAT vendors?
>
> A10Networks
> F5Networks
> Others?
>

Re: CGNAT Solutions

[Robert Blayzor]

On 4/29/20 10:29 AM, Mikael Abrahamsson wrote:
> There are some numbers in there for instance talking about 1024 ports
> per subscriber as a good number. In presentations I have seen over time,
> people typically talk about 512-4096 as being a good number for the bulk
> port allocation size.


So as a happy medium of about 2048 ports per subscriber, that's roughly
a 32:1 NAT/IP over-subscription ?

-- 
inoc.net!rblayzor
XMPP: rblayzor.AT.inoc.net
PGP:  https://pgp.inoc.net/rblayzor/

Re: CGNAT Solutions

[Mike Hammett]

I haven't used them, but 6-WIND is pretty proud of their CGNAT performance. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message -

From: "John Alcock"  
To: nanog@nanog.org 
Sent: Tuesday, April 28, 2020 2:12:29 PM 
Subject: CGNAT Solutions 


Afternoon, 


I run a small ISP in Tennessee. COVID has forced a lot of people to work from 
home. I am starting to run low on IP's and need to consider CGNAT. 


I do have IPV6 space, but we all know that until we force everyone to move to 
IPV6, we need to keep IPV4 up and running. 


I could buy more space, but I am really wondering if that is the best option. 
It is expensive. I know CGNAT devices are expensive as well, but it looks like 
I could stretch it out a bit. 


My thinking is to convert about 50% of my subscribers to CGNAT. 


I am interested in vendors or devices you have used in the past. I already know 
about the pitfalls many of my subscribers will have with CGNAT such as VPN's, 
Gamers, etc. 


What are your thoughts on CGNAT vendors? 


A10Networks 
F5Networks 
Others? 

Re: CGNAT Solutions

[Mikael Abrahamsson via NANOG]

On Wed, 29 Apr 2020, Robert Blayzor wrote:

One would think a 1000 ports would be enough, but if you have a dozen 
devices at home all browsing and doing various things, and with IOT, 
etc, maybe not?


https://www.juniper.net/documentation/en_US/junos/topics/concept/nat-best-practices.html

There are some numbers in there for instance talking about 1024 ports per 
subscriber as a good number. In presentations I have seen over time, 
people typically talk about 512-4096 as being a good number for the bulk 
port allocation size.


--
Mikael Abrahamssonemail: swm...@swm.pp.se

Re: CGNAT Solutions

[james jones]

How big is your ip pool for CGNAT?

On Wed, Apr 29, 2020 at 10:17 AM Robert Blayzor 
wrote:

> On 4/28/20 11:01 PM, Brandon Martin wrote:
> > Depending on how many IPs you need to reclaim and what your target
> > IP:subscriber ratio is, you may be able to eliminate the need for a lot
> > of logging by assigning a range of TCP/UDP ports to a single inside IP
> > so that the TCP/UDP port number implies a specific subscriber.
> >
> > You can't get rid of all the state tracking without also having the CPE
> > know which ports to use (in which case you might as well use LW4o6 or
> > MAP), but at least you can get it down to where you really only need to
> > log (or block and dole out public IPs as needed) port-less protocols.
>
>
> I'm wondering if there are any real world examples of this, namely in
> the realm of subscriber to IP and range of ports required, etc.  ie: Is
> is a range of 1000 ports enough for one residential subscriber? How
> about SMB where no global IP is required.
>
> One would think a 1000 ports would be enough, but if you have a dozen
> devices at home all browsing and doing various things, and with IOT,
> etc, maybe not?
>
>
> --
> inoc.net!rblayzor
> XMPP: rblayzor.AT.inoc.net
> PGP:  https://pgp.inoc.net/rblayzor/
>
-- 
Sent from Gmail Mobile

Re: CGNAT Solutions

[Tarko Tikan]

hey,


I'm wondering if there are any real world examples of this, namely in
the realm of subscriber to IP and range of ports required, etc.  ie: Is
is a range of 1000 ports enough for one residential subscriber? How
about SMB where no global IP is required.

One would think a 1000 ports would be enough, but if you have a dozen
devices at home all browsing and doing various things, and with IOT,
etc, maybe not?


1000 ports doesn't mean you can have at max 1000 layer-4 sessions at 
once. It means you can have 1000 sessions to single destination IP+port. 
You can reuse same source port numbers for different destination IP or 
even destination port.


We are seeing very good results with 256 ports per subscriber in the 
mobile scenario where consumer is mobile handset. So not directly 
translatable to broadband setup but still good datapoint.


If you must go CGNAT today it's only reasonable to use PBA (so you log 
only block allocations) or pure deterministic where you have strict 
mapping between inside IP and outside IP+portrange so you don't need any 
logs at all.


--
tarko

Re: CGNAT Solutions

[Ca By]

On Wed, Apr 29, 2020 at 1:06 AM Masataka Ohta <
mo...@necom830.hpcl.titech.ac.jp> wrote:

> Brandon Martin wrote:
>
> >> If you mean getting rid of logging, not necessarily. It is enough if
> >> CPEs are statically allocated ranges of external port numbers.
> >
> > Yes, you can get rid of the logging by statically allocating ranges of
> > port numbers to a particular customer.
>
> And, that was the original concern.
>
> > What I was referring to, though, was the programmatic state tracking of
> > the {external IP, external port}-{internal IP, internal port} mappings.
>
> OK.
>
> >   You can't eliminate that unless the CPE also knows what internal port
> > range it's mapped to so that it restricts what range it uses.  If you
> > can do that, you can get rid of the programmatic state tracking entirely
> > and just use static translations for TCP and UDP which, while nice, is
> > impractical.  You're about 95% of the way to LW4o6 or MAP at that point.
>
> Interesting. Then, if you can LW4o6 or MAP, you are about 95% of the
> way to E2ENAT with complete end to end transparency using IPv4 only,
> which means we don't need IPv6 with 4to6 NAT lacking the transparency.
>
> https://tools.ietf.org/html/draft-ohta-e2e-nat-00
>
> Masataka Ohta
>

Since we are talking numbers ans hard facts

42% of usa accesses google on ipv6

https://www.google.com/intl/en/ipv6/statistics.html




>

Re: CGNAT Solutions

[Robert Blayzor]

On 4/28/20 11:01 PM, Brandon Martin wrote:
> Depending on how many IPs you need to reclaim and what your target
> IP:subscriber ratio is, you may be able to eliminate the need for a lot
> of logging by assigning a range of TCP/UDP ports to a single inside IP
> so that the TCP/UDP port number implies a specific subscriber.
> 
> You can't get rid of all the state tracking without also having the CPE
> know which ports to use (in which case you might as well use LW4o6 or
> MAP), but at least you can get it down to where you really only need to
> log (or block and dole out public IPs as needed) port-less protocols.


I'm wondering if there are any real world examples of this, namely in
the realm of subscriber to IP and range of ports required, etc.  ie: Is
is a range of 1000 ports enough for one residential subscriber? How
about SMB where no global IP is required.

One would think a 1000 ports would be enough, but if you have a dozen
devices at home all browsing and doing various things, and with IOT,
etc, maybe not?


-- 
inoc.net!rblayzor
XMPP: rblayzor.AT.inoc.net
PGP:  https://pgp.inoc.net/rblayzor/

Re: CGNAT Solutions

[Masataka Ohta]

Brandon Martin wrote:


If you mean getting rid of logging, not necessarily. It is enough if
CPEs are statically allocated ranges of external port numbers.


Yes, you can get rid of the logging by statically allocating ranges of 
port numbers to a particular customer.


And, that was the original concern.

What I was referring to, though, was the programmatic state tracking of 
the {external IP, external port}-{internal IP, internal port} mappings.


OK.

  You can't eliminate that unless the CPE also knows what internal port 
range it's mapped to so that it restricts what range it uses.  If you 
can do that, you can get rid of the programmatic state tracking entirely 
and just use static translations for TCP and UDP which, while nice, is 
impractical.  You're about 95% of the way to LW4o6 or MAP at that point.


Interesting. Then, if you can LW4o6 or MAP, you are about 95% of the
way to E2ENAT with complete end to end transparency using IPv4 only,
which means we don't need IPv6 with 4to6 NAT lacking the transparency.

https://tools.ietf.org/html/draft-ohta-e2e-nat-00

Masataka Ohta

Re: CGNAT Solutions

[Brandon Martin]

On 4/29/20 2:35 AM, Masataka Ohta wrote:

If you mean getting rid of logging, not necessarily. It is enough if
CPEs are statically allocated ranges of external port numbers.


Yes, you can get rid of the logging by statically allocating ranges of  
port numbers to a particular customer.


What I was referring to, though, was the programmatic state tracking of  
the {external IP, external port}-{internal IP, internal port} mappings.  
 You can't eliminate that unless the CPE also knows what internal port  
range it's mapped to so that it restricts what range it uses.  If you  
can do that, you can get rid of the programmatic state tracking entirely  
and just use static translations for TCP and UDP which, while nice, is  
impractical.  You're about 95% of the way to LW4o6 or MAP at that point.

--
Brandon Martin

Re: CGNAT Solutions

[Masataka Ohta]

Brandon Martin wrote:

You can't get rid of all the state tracking without also having the CPE 
know which ports to use


If you mean getting rid of logging, not necessarily. It is enough if
CPEs are statically allocated ranges of external port numbers.

Masataka Ohta

Re: CGNAT Solutions

[Brandon Martin]

On 4/28/20 4:53 PM, William Herrin wrote:

How small is small? Up to a certain size regular NAT with enough
logging to trace back abusers will tend to work fine. if we're talking
single-digit gbps, it may not be worth the effort to consider the
wonderful world of CGNAT.


Depending on how many IPs you need to reclaim and what your target 
IP:subscriber ratio is, you may be able to eliminate the need for a lot 
of logging by assigning a range of TCP/UDP ports to a single inside IP 
so that the TCP/UDP port number implies a specific subscriber.


You can't get rid of all the state tracking without also having the CPE 
know which ports to use (in which case you might as well use LW4o6 or 
MAP), but at least you can get it down to where you really only need to 
log (or block and dole out public IPs as needed) port-less protocols.

--
Brandon Martin

Re: CGNAT Solutions

[Jared Geiger]

Take a look at DANOS for CG-NAT as a free solution or Netgate's TNSR has a
CG-NAT feature https://www.tnsr.com/features

On Tue, Apr 28, 2020 at 2:57 PM JORDI PALET MARTINEZ via NANOG <
nanog@nanog.org> wrote:

> I will say it is much better to consider 464XLAT with NAT64, if the CPEs
> allow it.
>
>
>
> https://datatracker.ietf.org/doc/rfc8683/
>
>
>
> I’m right now doing a deployment for 25.000.000 customers of an ISP (GPON,
> DLS and cellular mix), all the testing has been done, and all doing fine.
>
>
>
> I’ve done it already for smaller ISPs, but the size of this project is
> more interesting to better demonstrate that it just works.
>
>
>
> I plan to do a presentation when the information can be made public … bit
> delay because the Covid-19 confinement.
>
>
>
> Regards,
>
> Jordi
>
> @jordipalet
>
>
>
>
>
>
>
> El 28/4/20 21:15, "NANOG en nombre de John Alcock" <
> nanog-boun...@nanog.org en nombre de j...@alcock.org> escribió:
>
>
>
> Afternoon,
>
>
>
> I run a small ISP in Tennessee.  COVID has forced a lot of people to work
> from home.  I am starting to run low on IP's and need to consider CGNAT.
>
>
>
> I do have IPV6 space, but we all know that until we force everyone to move
> to IPV6, we need to keep IPV4 up and running.
>
>
>
> I could buy more space, but I am really wondering if that is the
> best option.  It is expensive. I know CGNAT devices are expensive as well,
> but it looks like I could stretch it out a bit.
>
>
>
> My thinking is to convert about 50% of my subscribers to CGNAT.
>
>
>
> I am interested in vendors or devices you have used in the past.  I
> already know about the pitfalls many of my subscribers will have with CGNAT
> such as VPN's, Gamers, etc.
>
>
>
> What are your thoughts on CGNAT vendors?
>
>
>
> A10Networks
>
> F5Networks
>
> Others?
>
> **
> IPv4 is over
> Are you ready for the new Internet ?
> http://www.theipv6company.com
> The IPv6 Company
>
> This electronic message contains information which may be privileged or
> confidential. The information is intended to be for the exclusive use of
> the individual(s) named above and further non-explicilty authorized
> disclosure, copying, distribution or use of the contents of this
> information, even if partially, including attached files, is strictly
> prohibited and will be considered a criminal offense. If you are not the
> intended recipient be aware that any disclosure, copying, distribution or
> use of the contents of this information, even if partially, including
> attached files, is strictly prohibited, will be considered a criminal
> offense, so you must reply to the original sender to inform about this
> communication and delete it.
>
>

Re: CGNAT Solutions

[JORDI PALET MARTINEZ via NANOG]

I will say it is much better to consider 464XLAT with NAT64, if the CPEs allow 
it.

 

https://datatracker.ietf.org/doc/rfc8683/

 

I’m right now doing a deployment for 25.000.000 customers of an ISP (GPON, DLS 
and cellular mix), all the testing has been done, and all doing fine.

 

I’ve done it already for smaller ISPs, but the size of this project is more 
interesting to better demonstrate that it just works.

 

I plan to do a presentation when the information can be made public … bit delay 
because the Covid-19 confinement.

 

Regards,

Jordi

@jordipalet

 

 

 

El 28/4/20 21:15, "NANOG en nombre de John Alcock"  escribió:

 

Afternoon,

 

I run a small ISP in Tennessee.  COVID has forced a lot of people to work from 
home.  I am starting to run low on IP's and need to consider CGNAT.

 

I do have IPV6 space, but we all know that until we force everyone to move to 
IPV6, we need to keep IPV4 up and running.

 

I could buy more space, but I am really wondering if that is the best option.  
It is expensive. I know CGNAT devices are expensive as well, but it looks like 
I could stretch it out a bit.

 

My thinking is to convert about 50% of my subscribers to CGNAT.

 

I am interested in vendors or devices you have used in the past.  I already 
know about the pitfalls many of my subscribers will have with CGNAT such as 
VPN's, Gamers, etc.

 

What are your thoughts on CGNAT vendors?  

 

A10Networks

F5Networks

Others?



**
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or 
confidential. The information is intended to be for the exclusive use of the 
individual(s) named above and further non-explicilty authorized disclosure, 
copying, distribution or use of the contents of this information, even if 
partially, including attached files, is strictly prohibited and will be 
considered a criminal offense. If you are not the intended recipient be aware 
that any disclosure, copying, distribution or use of the contents of this 
information, even if partially, including attached files, is strictly 
prohibited, will be considered a criminal offense, so you must reply to the 
original sender to inform about this communication and delete it.

Re: CGNAT Solutions

[William Herrin]

On Tue, Apr 28, 2020 at 12:12 PM John Alcock  wrote:
> I run a small ISP in Tennessee.  I am starting to run low on IP's and need to 
> consider CGNAT.

Hi John,

How small is small? Up to a certain size regular NAT with enough
logging to trace back abusers will tend to work fine. if we're talking
single-digit gbps, it may not be worth the effort to consider the
wonderful world of CGNAT.

Regards,
Bill Herrin

-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

RE: CGNAT Solutions

[Aaron Gould]

Hi John, I run a small/medium ISP in Texas.  A few years ago, needing to do the 
same thing you are speaking of, I lab evaluated the Cisco ASR9k VSM-500 and 
Juniper MX104 MS-MIC-16G… in the end I went with Juniper.  No regrets, been 
good and holding strong.  I’ve scaled it way beyond what I originally 
envisioned.  (but bought more as well)

 

I slow started my CGNat deployment, like with most things, baby-steps when 
doing something as extreme as taking away the public ip  address from my isp 
residential customers… so yeah, slow-start…

 

DSL was my first target.  One DSLAM at a time, waiting for issues to arise and 
dealing with them along the way, the best I could.  …until we had 6,000 dsl 
customers behind a pair of Juniper MX104’s with MS-MIC-16G cards, running fine. 
 (all done via mpls l3vpn for virtual L3 routing into and out of the nat 
boundary… so one vrf for inside, and one vrf for outside)…peak load as I recall 
was about 3 gbps on each MX104, so 6 gbps total.

 

Next, about a year or so later, we went after Cable Modem CMTS communities.  
But, added MS-MPC-128G modules to a pair of our mpls 100 gig ring MX960 nodes.  
This was another 5,000 subs or so.  (this was about 2 or 3 years ago).  Learned 
a lot during that one.  A lot about ecmp, inet.3 mp-ibgp route choices, (set 
protocols ldp track-igp-metric… is your friend), app, eim, eif, ams/mams 
interfaces and load-balancing on the source-ip…. Let that ride for a year or 
so…then…

 

…went after our FTTH communities.  Probably about 30 or 40 thousand ip’s were 
recoup’d here.  FTTH was nat’d behind (4) additional MS-MPC-128G modules in (4) 
other 100 gig mpls ring mx960 nodes.

 

There have been recent concerns about uPNP not working behind the cgnat’s.

 

All in all, we are getting lots of use out of our Juniper CGNat solution.  All 
told, it’s about 50,000 customers behind the (2) MX104’s and (6) MX960’s 
getting nat’d.

 

-Aaron

 

 

 

From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of John Alcock
Sent: Tuesday, April 28, 2020 2:12 PM
To: nanog@nanog.org
Subject: CGNAT Solutions

 

Afternoon,

 

I run a small ISP in Tennessee.  COVID has forced a lot of people to work from 
home.  I am starting to run low on IP's and need to consider CGNAT.

 

I do have IPV6 space, but we all know that until we force everyone to move to 
IPV6, we need to keep IPV4 up and running.

 

I could buy more space, but I am really wondering if that is the best option.  
It is expensive. I know CGNAT devices are expensive as well, but it looks like 
I could stretch it out a bit.

 

My thinking is to convert about 50% of my subscribers to CGNAT.

 

I am interested in vendors or devices you have used in the past.  I already 
know about the pitfalls many of my subscribers will have with CGNAT such as 
VPN's, Gamers, etc.

 

What are your thoughts on CGNAT vendors?  

 

A10Networks

F5Networks

Others?

Re: CGNAT Solutions

[Baldur Norddahl]

Just go with Linux and iptables. It is by far the cheapest option and it
just works.


tir. 28. apr. 2020 21.13 skrev John Alcock :

> Afternoon,
>
> I run a small ISP in Tennessee.  COVID has forced a lot of people to work
> from home.  I am starting to run low on IP's and need to consider CGNAT.
>
> I do have IPV6 space, but we all know that until we force everyone to move
> to IPV6, we need to keep IPV4 up and running.
>
> I could buy more space, but I am really wondering if that is the
> best option.  It is expensive. I know CGNAT devices are expensive as well,
> but it looks like I could stretch it out a bit.
>
> My thinking is to convert about 50% of my subscribers to CGNAT.
>
> I am interested in vendors or devices you have used in the past.  I
> already know about the pitfalls many of my subscribers will have with CGNAT
> such as VPN's, Gamers, etc.
>
> What are your thoughts on CGNAT vendors?
>
> A10Networks
> F5Networks
> Others?
>

Re: CGNAT

[Compton, Rich A]

Hi, I would suggest that you test fragmented traffic as well to your NAT 
device.  Fragment packets (that are not the first packet) don't have the L4 
info so the NAT device will have to keep them in memory until the first 
fragment comes in w/ the L4 info.  This can cause a DoS condition if the NAT 
device doesn't adequately prune fragmented packets from the memory when there 
is a flood of these type of packets.

On 2/7/19, 11:47 AM, "Aaron Gould"  wrote:

Rich, et al, 

Circling back on some older threads... I'm doing this because I've been
growing my cgnat environments and needing to remind myself of somethings,
etc...

If an attack is targeted at 1 ip address, you would think that if
would/could affect all the napt-44 (nat overloaded/pat'd) ip's that hide
behind it... but isn't that *IF* that traffic actually got through the nat
boundary and flowed to the intended target(s) ?

Unsolicited outside>inside traffic I believe results in a deny of
traffic... and I'm seeing that the nat actually builds those flows as drop
flows

I generated some traffic at a nat destination and I see all my traffic is
"Drop"... now I wonder if this is a fast path like in asic (pfe) hardware
being dropped... if so, it would seem that the nat boundary is yet a really
nice way to quickly drop unsolicited inbound traffic from perhaps bad
sources.

My source where I was generating traffic... Hollywood-ip (only works in the
movies) 256.256.191.133 (bad guy)

Nat destination where I sending traffic to... 256.256.130.4 (victim/target)

Now of course the resources/network outside the nat is bogged down, but the
inside nat domain seems to be unaffected in this case from what I can tell.

And again, I'm wondering if that "Drop" flow is lightweight/fast processing
for the ms-mpc-128g juniper gear ?


{master}
agould@960> show services sessions destination-prefix 256.256.130.4/32 |
grep 256.256.191.133 | refresh 1
---(refreshed at 2019-02-07 12:36:45 CST)---
---(refreshed at 2019-02-07 12:36:46 CST)---
---(refreshed at 2019-02-07 12:36:47 CST)---
---(refreshed at 2019-02-07 12:36:48 CST)---
---(refreshed at 2019-02-07 12:36:49 CST)---
---(refreshed at 2019-02-07 12:36:50 CST)---
---(refreshed at 2019-02-07 12:36:51 CST)---
---(refreshed at 2019-02-07 12:36:52 CST)---
TCP256.256.191.133:54519  ->  256.256.130.4:443Drop O
1
ICMP   256.256.191.133->  256.256.130.4Drop O
1
---(refreshed at 2019-02-07 12:36:53 CST)---
TCP256.256.191.133:54519  ->  256.256.130.4:443Drop O
1
ICMP   256.256.191.133->  256.256.130.4Drop O
1
---(refreshed at 2019-02-07 12:36:54 CST)---
TCP256.256.191.133:54519  ->  256.256.130.4:443Drop O
1
ICMP   256.256.191.133->  256.256.130.4Drop O
1
---(refreshed at 2019-02-07 12:36:55 CST)---
TCP256.256.191.133:54519  ->  256.256.130.4:443Drop O
1
ICMP   256.256.191.133->  256.256.130.4Drop O
1
---(refreshed at 2019-02-07 12:36:56 CST)---
---(refreshed at 2019-02-07 12:36:57 CST)---
---(refreshed at 2019-02-07 12:36:58 CST)---
UDP256.256.191.133:12998  ->  256.256.130.4:80 Drop O
1
UDP256.256.191.133:2  ->  256.256.130.4:80 Drop O
1
---(refreshed at 2019-02-07 12:36:59 CST)---
UDP256.256.191.133:12998  ->  256.256.130.4:80 Drop O
1
UDP256.256.191.133:2  ->  256.256.130.4:80 Drop O
1
---(refreshed at 2019-02-07 12:37:00 CST)---
UDP256.256.191.133:12998  ->  256.256.130.4:80 Drop O
1
UDP256.256.191.133:2  ->  256.256.130.4:80 Drop O
1
---(refreshed at 2019-02-07 12:37:01 CST)---
UDP256.256.191.133:12998  ->  256.256.130.4:80 Drop O
1
UDP256.256.191.133:2  ->  256.256.130.4:80 Drop O
1

- Aaron

-Original Message-
From: Compton, Rich A [mailto:rich.comp...@charter.com] 
Sent: Thursday, April 6, 2017 3:49 PM
To: Aaron Gould; 'Ahmed Munaf'; 'Nanog@Nanog'
Subject: Re: CGNAT

Hi Aaron, thanks for the info.  I¹m curious what you or others do about
DDoS attacks to CGNAT devices.  It seems that a single attack could affect
the thousands of customers that use those devices.  Also, do you have
issues detecting attacks vs. legitimate traffic when you have so much
traffic destined to a small group of IPs?

Rich Compton  |  Principal Eng |  314.596.2828
14810 Grass

RE: CGNAT

[Aaron Gould]

Rich, et al, 

Circling back on some older threads... I'm doing this because I've been
growing my cgnat environments and needing to remind myself of somethings,
etc...

If an attack is targeted at 1 ip address, you would think that if
would/could affect all the napt-44 (nat overloaded/pat'd) ip's that hide
behind it... but isn't that *IF* that traffic actually got through the nat
boundary and flowed to the intended target(s) ?

Unsolicited outside>inside traffic I believe results in a deny of
traffic... and I'm seeing that the nat actually builds those flows as drop
flows

I generated some traffic at a nat destination and I see all my traffic is
"Drop"... now I wonder if this is a fast path like in asic (pfe) hardware
being dropped... if so, it would seem that the nat boundary is yet a really
nice way to quickly drop unsolicited inbound traffic from perhaps bad
sources.

My source where I was generating traffic... Hollywood-ip (only works in the
movies) 256.256.191.133 (bad guy)

Nat destination where I sending traffic to... 256.256.130.4 (victim/target)

Now of course the resources/network outside the nat is bogged down, but the
inside nat domain seems to be unaffected in this case from what I can tell.

And again, I'm wondering if that "Drop" flow is lightweight/fast processing
for the ms-mpc-128g juniper gear ?


{master}
agould@960> show services sessions destination-prefix 256.256.130.4/32 |
grep 256.256.191.133 | refresh 1
---(refreshed at 2019-02-07 12:36:45 CST)---
---(refreshed at 2019-02-07 12:36:46 CST)---
---(refreshed at 2019-02-07 12:36:47 CST)---
---(refreshed at 2019-02-07 12:36:48 CST)---
---(refreshed at 2019-02-07 12:36:49 CST)---
---(refreshed at 2019-02-07 12:36:50 CST)---
---(refreshed at 2019-02-07 12:36:51 CST)---
---(refreshed at 2019-02-07 12:36:52 CST)---
TCP256.256.191.133:54519  ->  256.256.130.4:443Drop O
1
ICMP   256.256.191.133->  256.256.130.4Drop O
1
---(refreshed at 2019-02-07 12:36:53 CST)---
TCP256.256.191.133:54519  ->  256.256.130.4:443Drop O
1
ICMP   256.256.191.133->  256.256.130.4Drop O
1
---(refreshed at 2019-02-07 12:36:54 CST)---
TCP256.256.191.133:54519  ->  256.256.130.4:443Drop O
1
ICMP   256.256.191.133->  256.256.130.4Drop O
1
---(refreshed at 2019-02-07 12:36:55 CST)---
TCP256.256.191.133:54519  ->  256.256.130.4:443Drop O
1
ICMP   256.256.191.133->  256.256.130.4Drop O
1
---(refreshed at 2019-02-07 12:36:56 CST)---
---(refreshed at 2019-02-07 12:36:57 CST)---
---(refreshed at 2019-02-07 12:36:58 CST)---
UDP256.256.191.133:12998  ->  256.256.130.4:80 Drop O
1
UDP256.256.191.133:2  ->  256.256.130.4:80 Drop O
1
---(refreshed at 2019-02-07 12:36:59 CST)---
UDP256.256.191.133:12998  ->  256.256.130.4:80 Drop O
1
UDP256.256.191.133:2  ->  256.256.130.4:80 Drop O
1
---(refreshed at 2019-02-07 12:37:00 CST)---
UDP256.256.191.133:12998  ->  256.256.130.4:80 Drop O
1
UDP256.256.191.133:2  ->  256.256.130.4:80 Drop O
1
---(refreshed at 2019-02-07 12:37:01 CST)---
UDP256.256.191.133:12998  ->  256.256.130.4:80 Drop O
1
UDP256.256.191.133:2  ->  256.256.130.4:80 Drop O
1

- Aaron

-Original Message-
From: Compton, Rich A [mailto:rich.comp...@charter.com] 
Sent: Thursday, April 6, 2017 3:49 PM
To: Aaron Gould; 'Ahmed Munaf'; 'Nanog@Nanog'
Subject: Re: CGNAT

Hi Aaron, thanks for the info.  I¹m curious what you or others do about
DDoS attacks to CGNAT devices.  It seems that a single attack could affect
the thousands of customers that use those devices.  Also, do you have
issues detecting attacks vs. legitimate traffic when you have so much
traffic destined to a small group of IPs?

Rich Compton  |  Principal Eng |  314.596.2828
14810 Grasslands  Dr,Englewood,  CO80112

Re: cgnat - how do you handle customer issues

[Owen DeLong]

There’s also the issue of what a customer who needs something like GRE or IKE 
to work does from behind a CGNAT where there aren’t port numbers available for 
multiplexing.

Owen

> On Feb 27, 2018, at 2:42 PM, Lee Howard <l...@asgard.org> wrote:
> 
> 
> 
> On 02/27/2018 12:52 PM, Aaron Gould wrote:
>> Thanks
>> 
>>  
>> For #2 – what if the ports allocated aren’t enough for the amount of inet 
>> traffic the customer site uses ?  …is the customer denied service based on 
>> insufficient port range ?  …or are they assigned another block within that 
>> some ip’s range of I think it’s 0-64k or 1025-64k… but how far can you take 
>> that before there aren’t anymore port blocks left on that single ip ?  …and 
>> if you have to allocate that customer another port block from a *different* 
>> ip, then we are in the situation of the bank website not liking the fact 
>> that the session is bouncing to a different ip maybe ?
> 
> 
> Yes, common problem, and the session just fails (TCY SYN dropped) because of 
> insufficient ports.
> Most CGNs allow you to configure a range of ports for a customer, and reserve 
> an additional range if they need more ports. Yes, if you allocate 1000 ports 
> for one IPv4 address to each of 50 customers and they all need hundreds more 
> ports than that, you're going to run out of ports and connections fail.
> 
> This is why you have IPv6. Even while many web sites and apps don't support 
> IPv4, enough do that it relieves some pressure on your CGN.
> 
> Lee
>>  
>> - Aaron
>> 
>>  
>> From: Michael Crapse [mailto:mich...@wi-fiber.io]
>> Sent: Tuesday, February 27, 2018 11:19 AM
>> To: Mike Hammett
>> Cc: Aaron Gould; NANOG list
>> Subject: Re: cgnat - how do you handle customer issues
>> 
>>  
>> For number 2, I'm a fan of what mike suggests. I believe the technical term 
>> is MAP-T.
>> For number 1, anyone who wants one, gets one. We provide free public static 
>> IP to any customer who asks for one. Another solution, using above solution 
>> is to ask them which ports they need, and forward those to them using a port 
>> within their assign range. i.e. teach them how to access their home web 
>> server using a different port(say 32424, or similar). This won't solve all 
>> the issues, which is why we use solution 1.
>> 
>>  
>> On 27 February 2018 at 09:32, Mike Hammett <na...@ics-il.net> wrote:
>> 
>> I'm a fan of nailing each customer IP to a particular range of ports on a 
>> given public IP. Real easy to track who did what and to prevent shifting IPs.
>> 
>> 
>> 
>> 
>> -
>> Mike Hammett
>> Intelligent Computing Solutions
>> 
>> Midwest Internet Exchange
>> 
>> The Brothers WISP
>> 
>> - Original Message -
>> 
>> From: "Aaron Gould" <aar...@gvtc.com>
>> To: Nanog@nanog.org
>> Sent: Tuesday, February 27, 2018 10:30:21 AM
>> Subject: cgnat - how do you handle customer issues
>> 
>> 
>> Couple questions please. When you put thousands of customers behind a cgnat
>> boundary, how do you all handle customer complaints about the following.
>> 
>> 
>> 
>> 1 - for external connectivity to the customers premise devices, not being
>> able to access web servers, web cameras, etc, in their premises?
>> 
>> 
>> 
>> 2 - from the premise natted device, when customers go to a university or
>> bank web site, how do you handle randomly changing ip addresses/ports that
>> may occur due to idle time and session tear-down in nat table such that the
>> bank website has issues with seeing your session ip change?
>> 
>> 
>> 
>> 
>> 
>> -Aaron
>> 
>> 
>> 
>>  
>> 

Re: cgnat - how do you handle customer issues

[Lee Howard]

On 02/27/2018 12:52 PM, Aaron Gould wrote:

Thanks

  


For #2 – what if the ports allocated aren’t enough for the amount of inet 
traffic the customer site uses ?  …is the customer denied service based on 
insufficient port range ?  …or are they assigned another block within that some 
ip’s range of I think it’s 0-64k or 1025-64k… but how far can you take that 
before there aren’t anymore port blocks left on that single ip ?  …and if you 
have to allocate that customer another port block from a *different* ip, then 
we are in the situation of the bank website not liking the fact that the 
session is bouncing to a different ip maybe ?



Yes, common problem, and the session just fails (TCY SYN dropped) 
because of insufficient ports.
Most CGNs allow you to configure a range of ports for a customer, and 
reserve an additional range if they need more ports. Yes, if you 
allocate 1000 ports for one IPv4 address to each of 50 customers and 
they all need hundreds more ports than that, you're going to run out of 
ports and connections fail.


This is why you have IPv6. Even while many web sites and apps don't 
support IPv4, enough do that it relieves some pressure on your CGN.


Lee
  


- Aaron

  


From: Michael Crapse [mailto:mich...@wi-fiber.io]
Sent: Tuesday, February 27, 2018 11:19 AM
To: Mike Hammett
Cc: Aaron Gould; NANOG list
Subject: Re: cgnat - how do you handle customer issues

  


For number 2, I'm a fan of what mike suggests. I believe the technical term is 
MAP-T.
For number 1, anyone who wants one, gets one. We provide free public static IP 
to any customer who asks for one. Another solution, using above solution is to 
ask them which ports they need, and forward those to them using a port within 
their assign range. i.e. teach them how to access their home web server using a 
different port(say 32424, or similar). This won't solve all the issues, which 
is why we use solution 1.

  


On 27 February 2018 at 09:32, Mike Hammett <na...@ics-il.net> wrote:

I'm a fan of nailing each customer IP to a particular range of ports on a given 
public IP. Real easy to track who did what and to prevent shifting IPs.




-
Mike Hammett
Intelligent Computing Solutions

Midwest Internet Exchange

The Brothers WISP

- Original Message -

From: "Aaron Gould" <aar...@gvtc.com>
To: Nanog@nanog.org
Sent: Tuesday, February 27, 2018 10:30:21 AM
Subject: cgnat - how do you handle customer issues


Couple questions please. When you put thousands of customers behind a cgnat
boundary, how do you all handle customer complaints about the following.



1 - for external connectivity to the customers premise devices, not being
able to access web servers, web cameras, etc, in their premises?



2 - from the premise natted device, when customers go to a university or
bank web site, how do you handle randomly changing ip addresses/ports that
may occur due to idle time and session tear-down in nat table such that the
bank website has issues with seeing your session ip change?





-Aaron



  

Re: cgnat - how do you handle customer issues

[Lee Howard]

On 02/27/2018 11:30 AM, Aaron Gould wrote:

Couple questions please. When you put thousands of customers behind a cgnat
boundary, how do you all handle customer complaints about the following.

  


1 - for external connectivity to the customers premise devices, not being
able to access web servers, web cameras, etc, in their premises?

For web servers, you gently remind them of their ToS, and offer to 
upgrade them to a business account.


You offer to upgrade them to native IPv4, for a fee, and suggest they 
contact the manufacturer to see why it doesn't support IPv6.

I've heard of PCP for this, but never seen it in production on CPE.

I saw an early prototype of DS-Lite that included a customer portal that 
would allow the customer to enable port forwarding, but I can't imagine 
trying to walk a customer through two layers of port forwarding.


Most of these consumer electronics devices long ago started using 
ICE/TURN/STUN, and/or the company's web portal as their external 
communication platform. Doesn't mean all of them do.




2 - from the premise natted device, when customers go to a university or
bank web site, how do you handle randomly changing ip addresses/ports that
may occur due to idle time and session tear-down in nat table such that the
bank website has issues with seeing your session ip change?

  



You can extend your idle timers, of course, but only so much before 
you're squatting on all your ports.
After that, it's down to explaining to the university, bank, etc. that 
they need IPv6 if they want their web site to be reachable. Or at least 
they need to extend their idle timers.


Lee



  


-Aaron

Re: cgnat - how do you handle customer issues

[Chris Gross]

I utilize A10 CGNAT that allows dynamic NAT logging, since we're in a similar 
boat of utilization.

This email has been sent from my phone. Please excuse any brevity, typos, or 
lack of formality.

From: Aaron Gould <aar...@gvtc.com>
Sent: Tuesday, February 27, 2018 12:54
To: 'Michael Crapse'; 'Mike Hammett'
Cc: 'NANOG list'
Subject: RE: cgnat - how do you handle customer issues

Thanks



For #2 – what if the ports allocated aren’t enough for the amount of inet 
traffic the customer site uses ?  …is the customer denied service based on 
insufficient port range ?  …or are they assigned another block within that some 
ip’s range of I think it’s 0-64k or 1025-64k… but how far can you take that 
before there aren’t anymore port blocks left on that single ip ?  …and if you 
have to allocate that customer another port block from a *different* ip, then 
we are in the situation of the bank website not liking the fact that the 
session is bouncing to a different ip maybe ?



- Aaron



From: Michael Crapse [mailto:mich...@wi-fiber.io]
Sent: Tuesday, February 27, 2018 11:19 AM
To: Mike Hammett
Cc: Aaron Gould; NANOG list
Subject: Re: cgnat - how do you handle customer issues



For number 2, I'm a fan of what mike suggests. I believe the technical term is 
MAP-T.
For number 1, anyone who wants one, gets one. We provide free public static IP 
to any customer who asks for one. Another solution, using above solution is to 
ask them which ports they need, and forward those to them using a port within 
their assign range. i.e. teach them how to access their home web server using a 
different port(say 32424, or similar). This won't solve all the issues, which 
is why we use solution 1.



On 27 February 2018 at 09:32, Mike Hammett <na...@ics-il.net> wrote:

I'm a fan of nailing each customer IP to a particular range of ports on a given 
public IP. Real easy to track who did what and to prevent shifting IPs.




-
Mike Hammett
Intelligent Computing Solutions

Midwest Internet Exchange

The Brothers WISP

- Original Message -

From: "Aaron Gould" <aar...@gvtc.com>
To: Nanog@nanog.org
Sent: Tuesday, February 27, 2018 10:30:21 AM
Subject: cgnat - how do you handle customer issues


Couple questions please. When you put thousands of customers behind a cgnat
boundary, how do you all handle customer complaints about the following.



1 - for external connectivity to the customers premise devices, not being
able to access web servers, web cameras, etc, in their premises?



2 - from the premise natted device, when customers go to a university or
bank web site, how do you handle randomly changing ip addresses/ports that
may occur due to idle time and session tear-down in nat table such that the
bank website has issues with seeing your session ip change?





-Aaron

RE: cgnat - how do you handle customer issues

[Aaron Gould]

Thanks

 

For #2 – what if the ports allocated aren’t enough for the amount of inet 
traffic the customer site uses ?  …is the customer denied service based on 
insufficient port range ?  …or are they assigned another block within that some 
ip’s range of I think it’s 0-64k or 1025-64k… but how far can you take that 
before there aren’t anymore port blocks left on that single ip ?  …and if you 
have to allocate that customer another port block from a *different* ip, then 
we are in the situation of the bank website not liking the fact that the 
session is bouncing to a different ip maybe ?

 

- Aaron

 

From: Michael Crapse [mailto:mich...@wi-fiber.io] 
Sent: Tuesday, February 27, 2018 11:19 AM
To: Mike Hammett
Cc: Aaron Gould; NANOG list
Subject: Re: cgnat - how do you handle customer issues

 

For number 2, I'm a fan of what mike suggests. I believe the technical term is 
MAP-T.
For number 1, anyone who wants one, gets one. We provide free public static IP 
to any customer who asks for one. Another solution, using above solution is to 
ask them which ports they need, and forward those to them using a port within 
their assign range. i.e. teach them how to access their home web server using a 
different port(say 32424, or similar). This won't solve all the issues, which 
is why we use solution 1.

 

On 27 February 2018 at 09:32, Mike Hammett <na...@ics-il.net> wrote:

I'm a fan of nailing each customer IP to a particular range of ports on a given 
public IP. Real easy to track who did what and to prevent shifting IPs.




-
Mike Hammett
Intelligent Computing Solutions

Midwest Internet Exchange

The Brothers WISP

- Original Message -

From: "Aaron Gould" <aar...@gvtc.com>
To: Nanog@nanog.org
Sent: Tuesday, February 27, 2018 10:30:21 AM
Subject: cgnat - how do you handle customer issues


Couple questions please. When you put thousands of customers behind a cgnat
boundary, how do you all handle customer complaints about the following.



1 - for external connectivity to the customers premise devices, not being
able to access web servers, web cameras, etc, in their premises?



2 - from the premise natted device, when customers go to a university or
bank web site, how do you handle randomly changing ip addresses/ports that
may occur due to idle time and session tear-down in nat table such that the
bank website has issues with seeing your session ip change?





-Aaron



 

Re: cgnat - how do you handle customer issues

[Michael Crapse]

For number 2, I'm a fan of what mike suggests. I believe the technical term
is MAP-T.
For number 1, anyone who wants one, gets one. We provide free public static
IP to any customer who asks for one. Another solution, using above solution
is to ask them which ports they need, and forward those to them using a
port within their assign range. i.e. teach them how to access their home
web server using a different port(say 32424, or similar). This won't solve
all the issues, which is why we use solution 1.

On 27 February 2018 at 09:32, Mike Hammett  wrote:

> I'm a fan of nailing each customer IP to a particular range of ports on a
> given public IP. Real easy to track who did what and to prevent shifting
> IPs.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
>
> Midwest Internet Exchange
>
> The Brothers WISP
>
> - Original Message -
>
> From: "Aaron Gould" 
> To: Nanog@nanog.org
> Sent: Tuesday, February 27, 2018 10:30:21 AM
> Subject: cgnat - how do you handle customer issues
>
> Couple questions please. When you put thousands of customers behind a cgnat
> boundary, how do you all handle customer complaints about the following.
>
>
>
> 1 - for external connectivity to the customers premise devices, not being
> able to access web servers, web cameras, etc, in their premises?
>
>
>
> 2 - from the premise natted device, when customers go to a university or
> bank web site, how do you handle randomly changing ip addresses/ports that
> may occur due to idle time and session tear-down in nat table such that the
> bank website has issues with seeing your session ip change?
>
>
>
>
>
> -Aaron
>
>
>

Re: cgnat - how do you handle customer issues

[Mike Hammett]

I'm a fan of nailing each customer IP to a particular range of ports on a given 
public IP. Real easy to track who did what and to prevent shifting IPs. 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Aaron Gould"  
To: Nanog@nanog.org 
Sent: Tuesday, February 27, 2018 10:30:21 AM 
Subject: cgnat - how do you handle customer issues 

Couple questions please. When you put thousands of customers behind a cgnat 
boundary, how do you all handle customer complaints about the following. 



1 - for external connectivity to the customers premise devices, not being 
able to access web servers, web cameras, etc, in their premises? 



2 - from the premise natted device, when customers go to a university or 
bank web site, how do you handle randomly changing ip addresses/ports that 
may occur due to idle time and session tear-down in nat table such that the 
bank website has issues with seeing your session ip change? 





-Aaron 

Re: CGNAT

[Tassos Chatzithomaoglou]

With a ~59% dual-stack percentage and a 8% ds-lite percentage (aka 67%
of our subscriber base has IPv6), we get around 40% of IPv6 traffic.

--
Tassos

Radu-Adrian Feurdean wrote on 10/4/2017 1:11 μμ:
> On Fri, Apr 7, 2017, at 20:03, Mikael Abrahamsson wrote:
>> On Fri, 7 Apr 2017, Max Tulyev wrote:
>>
>>> BTW, does somebody check how implementing a native IPv6 decrease actual
>>> load of CGNAT?
>> Reports are that 30-50% of traffic will be IPv6 when you enable dual 
>> stack. This would be traffic that will not traverse your CGNAT.
> My data on customers supposed to be 100% dual-stack (unless they
> explicitely disable IPv6 on their side, which some of them do) says 25%
> on best days. It used to be up to 35% in late 2015.
> For reason unknown, it was going slightly down during 2016, with a
> sudden extra decrease in january this year.
>

Re: CGNAT

[Radu-Adrian Feurdean]

On Fri, Apr 7, 2017, at 20:03, Mikael Abrahamsson wrote:
> On Fri, 7 Apr 2017, Max Tulyev wrote:
> 
> > BTW, does somebody check how implementing a native IPv6 decrease actual
> > load of CGNAT?
> 
> Reports are that 30-50% of traffic will be IPv6 when you enable dual 
> stack. This would be traffic that will not traverse your CGNAT.

My data on customers supposed to be 100% dual-stack (unless they
explicitely disable IPv6 on their side, which some of them do) says 25%
on best days. It used to be up to 35% in late 2015.
For reason unknown, it was going slightly down during 2016, with a
sudden extra decrease in january this year.

Re: CGNAT

[Ed Lopez]

A lot depends on the CGNAT features you are looking to support, some
considerations:

- Are you looking for port block allocation for bulk logging, where a given
subscriber is given a block of source TCP/UDP ports on a translated IP
address
- How many translations and session rate are you looking to support
- Do you require Port Control Protocol (PCP) support for inbound pinholing
reservations?  Do your subscribers support uPnP to PCP translation?
- Are you looking to support RFC 6598 (carrier use of 100.64.0.0/10 for
CGNAT)?
- Are you looking to support DS-Lite (RFC 6333) or lw4o6 (RFC 7596)?  Both
have significantly different requirements relative to CGNAT (DS-Lite
assumes translation of subscriber RFC 1918 addresses tracking their IPv6
address in the translation table, lw4o6 assumes translation from RFC 1918
to RFC 6598 at the subscriber/B4 prior to IPv6 encapsulation plus
translation of RFC 6598 to public at the CGNAT/AFTR)

Generally, I tend to recommend F5 BIG-IP from a CGNAT feature standpoint

- Ed

On Fri, Apr 7, 2017 at 1:48 PM, Aaron Gould  wrote:

> Thanks Rich, you bring up some good points.  Yes it would seem that an
> attack aimed at a target IP address would in-fact now have a greater
> surface
> since that IP address is being used by many people.  When we
> remotely-trigger-black-hole (RTBH) route an ip address (/32 host route)
> into
> a black hole to stop an attack you're right, now you've completed the
> ddos, not only for one customer, but hundreds or thousands that were using
> that public ip address through the NAT appliance.  ...to which I've told my
> NOC to not act on any of the /24's-worth of address space the we use for
> NAT.
>
> Interestingly, the nature of NAT is that it doesn't allow in-bound traffic
> unless a previous out-bound packet had been sent from customer-side to
> internet-side and caused the NAT translation to be built therefore, an
> outside-initiated DDoS attack would be automatically blocked by a NAT
> boundary*.  This would cause the DDoS to not go as far as it did in the
> non-nat scenario.   ...so with cgnat you've caused your reach of DDoS to be
> shortened.  ...but of course this doesn't cause the DDoS to not occur and
> to
> not reach the NAT boundary...the attack still arrives.  You have to
> continue
> with other layers of security, defense and mitigation in other areas/layers
> of your network.
>
> - Aaron
>
> * (I guess unless they were able to guess-spoof the exact ip address and
> port number of an existing nat session, but then it would seem that they
> would only reach that same port-address-translated session
> destination...which I think would be a single ip address endpoint and port
> number)
>
>
>


-- 

*Ed Lopez* | *Security Architect*
ed.lo...@corsa.com |11 Hines Rd (suite 203) | Ottawa, Ontario K2K 2X1 Canada
Mobile +1.703.220.0988  | www.corsa.com

*Provide line-rate mitigation against DDoS attacks using the Corsa NSE7000
Series. *

*Learn how to make your network better with Corsa Performance SDN
.  *


*This email and any attachments are intended for the sole use of the named
recipient(s) and contain(s) confidential information that may be
proprietary, privileged or copyrighted under applicable law. If you are not
the intended recipient, do not read, copy, or forward this email message or
any attachments and delete this message and any attachments immediately.*

Re: CGNAT

[Compton, Rich A]

Hi Aaron, thanks for the info.  I¹m curious what you or others do about
DDoS attacks to CGNAT devices.  It seems that a single attack could affect
the thousands of customers that use those devices.  Also, do you have
issues detecting attacks vs. legitimate traffic when you have so much
traffic destined to a small group of IPs?

Rich Compton  |  Principal Eng |  314.596.2828
14810 Grasslands  Dr,Englewood,  CO80112






On 4/6/17, 2:33 PM, "NANOG on behalf of Aaron Gould"
 wrote:

>Last year I evaluated Cisco ASR9006/VSM-500 and Juniper MX104/MS-MIC-16G
>in
>my lab.
>
>I went with MX104/MS-MIC-16G.  I love it.
>
>I deployed (2) MX104's.  Each MX104 has a single MX-MIC-16G card in it.  I
>integrated this CGNAT with MPLS L3VPN's for NAT Inside vrf and NAT outside
>vrf.  Both MX104's learn 0/0 route for outside and send a 0/0 route for
>inside to all the PE's that have DSLAMs connected to them.  So each PE
>with
>DSL connected to it learns default route towards 2 equal cost MX104's.  I
>could easily add a third MX104 to this modular architecture.
>
>I have 7,000 DSL broadband customers behind it.  Peak time throughput is
>hitting up at 4 gbps... I see a little over 100,000 service flows
>(translations) at peak time
>
>I think each MX104 MS-MIC-16G can able about ~7 million translations and
>about 7 gbps of cgnat throughput... so I'm good.
>
>I have a /25 for each MX104 outside public address pool (so /24 total for
>both MX104's)... pretty sweet how I use /24 for ~7,000 customers :)
>
>I'll freeze this probably for DSL and not put anything else behind it.  I
>want to leave well-enough alone.
>
>If I move forward with CGNAT'ing Cable Modem (~6,000 more subsrcibers)
>I'll
>probably roll-out (2) more MX104's with a new vrf for that...
>
>If I move forward with CGNAT'ing FTTH (~20,000 more subsrcibers) I'll
>probably roll-out (2) MX240/480/960 with MS-MPC... I feel I'd want/need
>something beefier for FTTH...
>
>- Aaron
>
>

E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for 
the addressee(s) and may contain confidential and/or legally privileged 
information. If you are not the intended recipient of this message or if this 
message has been addressed to you in error, please immediately alert the sender 
by reply e-mail and then delete this message and any attachments. If you are 
not the intended recipient, you are notified that any use, dissemination, 
distribution, copying, or storage of this message or any attachment is strictly 
prohibited.

Re: CGNAT

[Pshem Kowalczyk]

I can confirm that percentage (at least with residential customer base).
All big content providers and a number of CDNs will do IPv6 by default. One
thing that will heavily affect this is the CPE equipment (which might not
have IPv6 enabled or even be capable of it).

kind regards
Pshem


On Sat, 8 Apr 2017 at 06:19 Mikael Abrahamsson  wrote:

> On Fri, 7 Apr 2017, Max Tulyev wrote:
>
> > BTW, does somebody check how implementing a native IPv6 decrease actual
> > load of CGNAT?
>
> Reports are that 30-50% of traffic will be IPv6 when you enable dual
> stack. This would be traffic that will not traverse your CGNAT.
>
> --
> Mikael Abrahamssonemail: swm...@swm.pp.se
>

RE: CGNAT

[Aaron Gould]

Thanks Max, I've thought about that and tested some ipv6 (6vpe, mpls l3vpn
w/ipv6 dual stacked) in my network.

In my CGNAT testing for my 7,000 dsl customers, I've already tested the
inter-vrf route leaks that will be required for ipv6-flow-around to bypass
the IPv4 CGNAT boundary so, I have tested dual stacking my dsl customers
with v4/v6 and seen that the v4 does flow via cgnat and v6 does bypass nat.
I could dig up some ios(xr)/junos inter-vrf route leak/policy configs if
anyone could benefit from that.

I'm actually anxious to get started on my ipv6 deployment in my dsl space...
as you have eluded to Max, this would push out the life of my ipv4 cgnat
juniper mx104/ms-mic-16g boundary since we would expect the ipv6 traffic to
flow naturally to my internet pipes un-natted and thus relive the nat nodes.
...and as more and more ipv6 is adopted in the world, the nat44/napt cgnat
boundary is less and less needed until ultimately, we are in a ipv6-only
world.

-Aaron

Re: CGNAT

[Mikael Abrahamsson]

On Fri, 7 Apr 2017, Max Tulyev wrote:


BTW, does somebody check how implementing a native IPv6 decrease actual
load of CGNAT?


Reports are that 30-50% of traffic will be IPv6 when you enable dual 
stack. This would be traffic that will not traverse your CGNAT.


--
Mikael Abrahamssonemail: swm...@swm.pp.se

Re: CGNAT

[Max Tulyev]

BTW, does somebody check how implementing a native IPv6 decrease actual
load of CGNAT?


On 06.04.17 23:33, Aaron Gould wrote:
> Last year I evaluated Cisco ASR9006/VSM-500 and Juniper MX104/MS-MIC-16G in
> my lab.
> 
> I went with MX104/MS-MIC-16G.  I love it.
> 
> I deployed (2) MX104's.  Each MX104 has a single MX-MIC-16G card in it.  I
> integrated this CGNAT with MPLS L3VPN's for NAT Inside vrf and NAT outside
> vrf.  Both MX104's learn 0/0 route for outside and send a 0/0 route for
> inside to all the PE's that have DSLAMs connected to them.  So each PE with
> DSL connected to it learns default route towards 2 equal cost MX104's.  I
> could easily add a third MX104 to this modular architecture.
> 
> I have 7,000 DSL broadband customers behind it.  Peak time throughput is
> hitting up at 4 gbps... I see a little over 100,000 service flows
> (translations) at peak time
> 
> I think each MX104 MS-MIC-16G can able about ~7 million translations and
> about 7 gbps of cgnat throughput... so I'm good.
> 
> I have a /25 for each MX104 outside public address pool (so /24 total for
> both MX104's)... pretty sweet how I use /24 for ~7,000 customers :) 
> 
> I'll freeze this probably for DSL and not put anything else behind it.  I
> want to leave well-enough alone.
> 
> If I move forward with CGNAT'ing Cable Modem (~6,000 more subsrcibers) I'll
> probably roll-out (2) more MX104's with a new vrf for that...
> 
> If I move forward with CGNAT'ing FTTH (~20,000 more subsrcibers) I'll
> probably roll-out (2) MX240/480/960 with MS-MPC... I feel I'd want/need
> something beefier for FTTH...
> 
> - Aaron
> 
> 
> 

RE: CGNAT

[Aaron Gould]

Thanks Rich, you bring up some good points.  Yes it would seem that an
attack aimed at a target IP address would in-fact now have a greater surface
since that IP address is being used by many people.  When we
remotely-trigger-black-hole (RTBH) route an ip address (/32 host route) into
a black hole to stop an attack you're right, now you've completed the
ddos, not only for one customer, but hundreds or thousands that were using
that public ip address through the NAT appliance.  ...to which I've told my
NOC to not act on any of the /24's-worth of address space the we use for
NAT.

Interestingly, the nature of NAT is that it doesn't allow in-bound traffic
unless a previous out-bound packet had been sent from customer-side to
internet-side and caused the NAT translation to be built therefore, an
outside-initiated DDoS attack would be automatically blocked by a NAT
boundary*.  This would cause the DDoS to not go as far as it did in the
non-nat scenario.   ...so with cgnat you've caused your reach of DDoS to be
shortened.  ...but of course this doesn't cause the DDoS to not occur and to
not reach the NAT boundary...the attack still arrives.  You have to continue
with other layers of security, defense and mitigation in other areas/layers
of your network.

- Aaron

* (I guess unless they were able to guess-spoof the exact ip address and
port number of an existing nat session, but then it would seem that they
would only reach that same port-address-translated session
destination...which I think would be a single ip address endpoint and port
number)

RE: CGNAT

[Aaron Gould]

Last year I evaluated Cisco ASR9006/VSM-500 and Juniper MX104/MS-MIC-16G in
my lab.

I went with MX104/MS-MIC-16G.  I love it.

I deployed (2) MX104's.  Each MX104 has a single MX-MIC-16G card in it.  I
integrated this CGNAT with MPLS L3VPN's for NAT Inside vrf and NAT outside
vrf.  Both MX104's learn 0/0 route for outside and send a 0/0 route for
inside to all the PE's that have DSLAMs connected to them.  So each PE with
DSL connected to it learns default route towards 2 equal cost MX104's.  I
could easily add a third MX104 to this modular architecture.

I have 7,000 DSL broadband customers behind it.  Peak time throughput is
hitting up at 4 gbps... I see a little over 100,000 service flows
(translations) at peak time

I think each MX104 MS-MIC-16G can able about ~7 million translations and
about 7 gbps of cgnat throughput... so I'm good.

I have a /25 for each MX104 outside public address pool (so /24 total for
both MX104's)... pretty sweet how I use /24 for ~7,000 customers :) 

I'll freeze this probably for DSL and not put anything else behind it.  I
want to leave well-enough alone.

If I move forward with CGNAT'ing Cable Modem (~6,000 more subsrcibers) I'll
probably roll-out (2) more MX104's with a new vrf for that...

If I move forward with CGNAT'ing FTTH (~20,000 more subsrcibers) I'll
probably roll-out (2) MX240/480/960 with MS-MPC... I feel I'd want/need
something beefier for FTTH...

- Aaron

Re: CGNAT

[Shahab Vahabzadeh]

Hello Ahmad,I am using F5 for CGNAT, right now 250K subscriber 
with 28Gbps bandwidth, I will double it with the second appliance easily 
soon.Its high performance and I like it.Any time Any QuestionThanks


Ch,
Shahab






On Thu, Apr 6, 2017 at 7:41 PM +0430, "Ahmed Munaf"  
wrote:











Hi, 

Any recommendation regarding CGNAT appliance who try it and which brand is the 
best from his perspective!

The throughput which I want to pass through the CGNAT is about 40Gbits and 
number of subscribers are about 40,000 subscribers. 


Regards,
Ahmed 

Re: CGNAT - Seeking Real World Experience

[Tassos Chatzithomaoglou]

I had given some numbers for PBA in 
http://puck.nether.net/pipermail/cisco-nsp/2016-February/101908.html

--
Tassos

Adam wrote on 23/11/16 23:17:
> I'm crunching the numbers on the cost effectiveness of implementing CGN vs
> IPv4 auctions. The determining factor is how many ephemeral ports are
> reserved for each customer. This is for a residential broadband environment.
>
> Is anybody doing deterministic NAT/PAT (i.e. each customer gets X ports -
> no more, no less)? My thinking is that X=8192 would cover even the power
> users. However, with only 8 customers per public IPv4 address, CGN is not
> worth the trouble. With X=8192, our money and time would better be spent
> acquiring additional IPv4 space. Are people successfully using a smaller
> deterministic port allocation? What's your X?
>
> If I can't make the numbers work for deterministic NAT, I might be able to
> live with dynamic port assignments. Specifically, I'm referring to what
> vendor J calls "Port Block Allocation". I was thinking 1024 ports per
> block, with up to 8 blocks per customer (and a bunch of log correlation to
> determine who was using which ip:port tuple at a given datetime). I *can*
> make the math work out in favor of CGN if the average customer uses <= 3072
> ports (3 blocks). But is that going to be enough? I'd love to hear other
> people's experiences.
>
> Thanks!
> -Adam
>

Re: CGNAT - Seeking Real World Experience

[Stepan Kucherenko]

Don't try detereministic NAT, it's not worth it. You'll waste a lot of 
port capacity on most users, and it might still be problematic for power 
users.


Just try to match one user to one real IP, many sites/applications don't 
like when there are several requests from one user with different IPs. 
After that just stack as many users on one IP as you can and that's it. 
It's the only way CGN can be worth the trouble.



If you really need to know who was using which port, just log them and 
correlate them when/if the need arises.



On 24.11.2016 00:17, Adam wrote:

I'm crunching the numbers on the cost effectiveness of implementing CGN vs
IPv4 auctions. The determining factor is how many ephemeral ports are
reserved for each customer. This is for a residential broadband environment.

Is anybody doing deterministic NAT/PAT (i.e. each customer gets X ports -
no more, no less)? My thinking is that X=8192 would cover even the power
users. However, with only 8 customers per public IPv4 address, CGN is not
worth the trouble. With X=8192, our money and time would better be spent
acquiring additional IPv4 space. Are people successfully using a smaller
deterministic port allocation? What's your X?

If I can't make the numbers work for deterministic NAT, I might be able to
live with dynamic port assignments. Specifically, I'm referring to what
vendor J calls "Port Block Allocation". I was thinking 1024 ports per
block, with up to 8 blocks per customer (and a bunch of log correlation to
determine who was using which ip:port tuple at a given datetime). I *can*
make the math work out in favor of CGN if the average customer uses <= 3072
ports (3 blocks). But is that going to be enough? I'd love to hear other
people's experiences.

Thanks!
-Adam

Re: CGNAT - Seeking Real World Experience

[Ca By]

On Thu, Nov 24, 2016 at 7:05 PM Adam  wrote:

> I'm crunching the numbers on the cost effectiveness of implementing CGN vs
> IPv4 auctions. The determining factor is how many ephemeral ports are
> reserved for each customer. This is for a residential broadband
> environment.
>
> Is anybody doing deterministic NAT/PAT (i.e. each customer gets X ports -
> no more, no less)? My thinking is that X=8192 would cover even the power
> users. However, with only 8 customers per public IPv4 address, CGN is not
> worth the trouble. With X=8192, our money and time would better be spent
> acquiring additional IPv4 space. Are people successfully using a smaller
> deterministic port allocation? What's your X?
>
> If I can't make the numbers work for deterministic NAT, I might be able to
> live with dynamic port assignments. Specifically, I'm referring to what
> vendor J calls "Port Block Allocation". I was thinking 1024 ports per
> block, with up to 8 blocks per customer (and a bunch of log correlation to
> determine who was using which ip:port tuple at a given datetime). I *can*
> make the math work out in favor of CGN if the average customer uses <= 3072
> ports (3 blocks). But is that going to be enough? I'd love to hear other
> people's experiences.
>
> Thanks!
> -Adam
>


We see around 70% of traffic using ipv6 (goog, fb, netflix, ... now
cloudfront and Cloudflare ) , that takes a lot of the sting out of CGN
cost.

CB