Re: IPv6 in Shared guest network

[Wido den Hollander]

Op 06/12/2023 om 09:45 schreef Tobias Rehn:

Hey Wido,

thank you. That solves a lot of trouble in my head. :) The documentation 
is not that clear.


IPv6 ist a must have for us

We are using Juniper MX gear as gateways. I just have to check if we can 
do SLAAC with our current JUNOS version and setup as we are using 
EVPN+MPLS with all-active multihoming on the gateway side.




Yes, that is possible. You need to set the 'protocol 
router-advertisement' on your IRB device for your EVPN environment.


You probably want to use Anycast IPv6 gateways in this case and 
configure the RA to be send from the Juniper not to be the Link-Local 
address but the Anycast address of the router(s).


Wido



Am Mi., 6. Dez. 2023 um 09:06 Uhr schrieb Wido den Hollander 
mailto:w...@denhollander.io>>:


Hi,

Great to see you are deploying IPv6!

Op 05/12/2023 om 16:05 schreef Tobias Rehn:
 > Hey together,
 >
 > I am in the process of setting up IPv6 in a shared guest network.
I have
 > successfully added the network with both IPv4 and IPv6 address space.
 >
 > For IPv4 the whole thing is working perfectly but I am failing
with IPv6. I
 > read the documentation and for me it is somehow unclear how the IPv6
 > addresses are installed on an instance.
 >
 > Speaking about the following document:
 > http://docs.cloudstack.apache.org/en/4.18.1.0/plugins/ipv6.html

 >
 > In the documentation it says about Shared Networks... "The user VM
 > generates an IPv6 link local address by itself, and gets an IPv6
global or
 > site local address through DHCPv6."
 >

You should enable IPv6 Router Advertisements so your router, not the
VR,
advertises itself on the network as a gateway. In this message it
should
also announce the prefix (usually /64) to be used in that network.

Based on the IPv6 Prefix and it's MAC address the Instance will
obtain a
unique address.

Make sure that IPv6 privacy extentensions are disabled inside the VM
template.

 > But then further down... "The gateway of the guest network
generates Router
 > Advisement and Response messages to Router Solicitation."
 >
 > Does the VR handle the DHCP6 requests? Or should this be handled
by our
 > gateways?
 >

DHCP is not used with IPv6, it's all using Router Advertisements and
SLAAC, StateLess Address Auto Configuration.

What routers are you using for the shared network?

Wido

 > Hopyfully, someone can point me to the right direction. Thanks.
 >
 >
 > Best
 > Tobias
 >

RE: IPv6 in Shared guest network

[Alex Mattioli]

Hi Tobias,

I'm not sure if it fits your use case (and Wido already gave you all the 
necessary advice on Shared networks and IPv6), but I'd like to highlight that 
there's support for IPv6 in Isolated Networks and VPCs as well.

Cheers
Alex

 


-Original Message-
From: Tobias Rehn  
Sent: Wednesday, December 6, 2023 9:45 AM
To: Wido den Hollander 
Cc: users@cloudstack.apache.org
Subject: Re: IPv6 in Shared guest network

Hey Wido,

thank you. That solves a lot of trouble in my head. :) The documentation is not 
that clear.

IPv6 ist a must have for us

We are using Juniper MX gear as gateways. I just have to check if we can do 
SLAAC with our current JUNOS version and setup as we are using EVPN+MPLS with 
all-active multihoming on the gateway side.


Am Mi., 6. Dez. 2023 um 09:06 Uhr schrieb Wido den Hollander <
w...@denhollander.io>:

> Hi,
>
> Great to see you are deploying IPv6!
>
> Op 05/12/2023 om 16:05 schreef Tobias Rehn:
> > Hey together,
> >
> > I am in the process of setting up IPv6 in a shared guest network. I 
> > have successfully added the network with both IPv4 and IPv6 address space.
> >
> > For IPv4 the whole thing is working perfectly but I am failing with
> IPv6. I
> > read the documentation and for me it is somehow unclear how the IPv6 
> > addresses are installed on an instance.
> >
> > Speaking about the following document:
> > http://docs.cloudstack.apache.org/en/4.18.1.0/plugins/ipv6.html
> >
> > In the documentation it says about Shared Networks... "The user VM 
> > generates an IPv6 link local address by itself, and gets an IPv6 
> > global
> or
> > site local address through DHCPv6."
> >
>
> You should enable IPv6 Router Advertisements so your router, not the 
> VR, advertises itself on the network as a gateway. In this message it 
> should also announce the prefix (usually /64) to be used in that network.
>
> Based on the IPv6 Prefix and it's MAC address the Instance will obtain 
> a unique address.
>
> Make sure that IPv6 privacy extentensions are disabled inside the VM 
> template.
>
> > But then further down... "The gateway of the guest network generates
> Router
> > Advisement and Response messages to Router Solicitation."
> >
> > Does the VR handle the DHCP6 requests? Or should this be handled by 
> > our gateways?
> >
>
> DHCP is not used with IPv6, it's all using Router Advertisements and 
> SLAAC, StateLess Address Auto Configuration.
>
> What routers are you using for the shared network?
>
> Wido
>
> > Hopyfully, someone can point me to the right direction. Thanks.
> >
> >
> > Best
> > Tobias
> >
>

Re: IPv6 in Shared guest network

[Tobias Rehn]

Hey Wido,

thank you. That solves a lot of trouble in my head. :) The documentation is
not that clear.

IPv6 ist a must have for us

We are using Juniper MX gear as gateways. I just have to check if we can do
SLAAC with our current JUNOS version and setup as we are using EVPN+MPLS
with all-active multihoming on the gateway side.


Am Mi., 6. Dez. 2023 um 09:06 Uhr schrieb Wido den Hollander <
w...@denhollander.io>:

> Hi,
>
> Great to see you are deploying IPv6!
>
> Op 05/12/2023 om 16:05 schreef Tobias Rehn:
> > Hey together,
> >
> > I am in the process of setting up IPv6 in a shared guest network. I have
> > successfully added the network with both IPv4 and IPv6 address space.
> >
> > For IPv4 the whole thing is working perfectly but I am failing with
> IPv6. I
> > read the documentation and for me it is somehow unclear how the IPv6
> > addresses are installed on an instance.
> >
> > Speaking about the following document:
> > http://docs.cloudstack.apache.org/en/4.18.1.0/plugins/ipv6.html
> >
> > In the documentation it says about Shared Networks... "The user VM
> > generates an IPv6 link local address by itself, and gets an IPv6 global
> or
> > site local address through DHCPv6."
> >
>
> You should enable IPv6 Router Advertisements so your router, not the VR,
> advertises itself on the network as a gateway. In this message it should
> also announce the prefix (usually /64) to be used in that network.
>
> Based on the IPv6 Prefix and it's MAC address the Instance will obtain a
> unique address.
>
> Make sure that IPv6 privacy extentensions are disabled inside the VM
> template.
>
> > But then further down... "The gateway of the guest network generates
> Router
> > Advisement and Response messages to Router Solicitation."
> >
> > Does the VR handle the DHCP6 requests? Or should this be handled by our
> > gateways?
> >
>
> DHCP is not used with IPv6, it's all using Router Advertisements and
> SLAAC, StateLess Address Auto Configuration.
>
> What routers are you using for the shared network?
>
> Wido
>
> > Hopyfully, someone can point me to the right direction. Thanks.
> >
> >
> > Best
> > Tobias
> >
>

Re: IPv6 in Shared guest network

[Wido den Hollander]

Hi,

Great to see you are deploying IPv6!

Op 05/12/2023 om 16:05 schreef Tobias Rehn:

Hey together,

I am in the process of setting up IPv6 in a shared guest network. I have
successfully added the network with both IPv4 and IPv6 address space.

For IPv4 the whole thing is working perfectly but I am failing with IPv6. I
read the documentation and for me it is somehow unclear how the IPv6
addresses are installed on an instance.

Speaking about the following document:
http://docs.cloudstack.apache.org/en/4.18.1.0/plugins/ipv6.html

In the documentation it says about Shared Networks... "The user VM
generates an IPv6 link local address by itself, and gets an IPv6 global or
site local address through DHCPv6."



You should enable IPv6 Router Advertisements so your router, not the VR, 
advertises itself on the network as a gateway. In this message it should 
also announce the prefix (usually /64) to be used in that network.


Based on the IPv6 Prefix and it's MAC address the Instance will obtain a 
unique address.


Make sure that IPv6 privacy extentensions are disabled inside the VM 
template.



But then further down... "The gateway of the guest network generates Router
Advisement and Response messages to Router Solicitation."

Does the VR handle the DHCP6 requests? Or should this be handled by our
gateways?



DHCP is not used with IPv6, it's all using Router Advertisements and 
SLAAC, StateLess Address Auto Configuration.


What routers are you using for the shared network?

Wido


Hopyfully, someone can point me to the right direction. Thanks.


Best
Tobias

Re: IPv6 error :: ACS 4.16.1

[Wido den Hollander]

Op 08-04-2022 om 21:17 schreef Ranjit Jadhav:

Thank you for your response, i was able to create IPv6 networks using both
methods. It is showing IPv6 IP assigned to NIC as well in ACS. But not able
to use it yet. VM was created using CentOS7.

Do I need to manually add IPv6 ips in ifcfg-eth0? Any more guidance for
actual integration with OS will be of great help.



Is your router sending our Router Advertisements with the prefix to be 
used in that subnet? If not, please make sure it does.


The Linux VMs should be set to autoconfigure with Privacy Extensions 
disabled.


Wido


Thanks again

Ranjit

On Mon, 28 Mar 2022 at 15:01, Wei ZHOU  wrote:


Hi Ranjit,

Currently CloudStack virtual routers must configure an IPv4 address on each
interface. Therefore you cannot create shared networks with only IPv6.
However, the default network offering for shared networks supports
dhcp/dns/userdata services which requires a CloudStack VR.

Solutions
(1) create a network with dummy(e.g. private) IPv4 addresses. OR
(2) Create a network offering without any services, then create a shared
network with the new offering.

-Wei

On Mon, 28 Mar 2022 at 11:23, Ranjit Jadhav 
wrote:


Hello,

I am getting the following error while trying to create a shared network
for IPv6, recently upgraded to 4.16.1

"Currently IPv6-only Shared network with Virtual Router provider is not
supported."

Any guideline will be of great help.

Thanks and regards,
Ranjit

Re: IPv6 error :: ACS 4.16.1

[Ranjit Jadhav]

Thank you for your response, i was able to create IPv6 networks using both
methods. It is showing IPv6 IP assigned to NIC as well in ACS. But not able
to use it yet. VM was created using CentOS7.

Do I need to manually add IPv6 ips in ifcfg-eth0? Any more guidance for
actual integration with OS will be of great help.

Thanks again

Ranjit

On Mon, 28 Mar 2022 at 15:01, Wei ZHOU  wrote:

> Hi Ranjit,
>
> Currently CloudStack virtual routers must configure an IPv4 address on each
> interface. Therefore you cannot create shared networks with only IPv6.
> However, the default network offering for shared networks supports
> dhcp/dns/userdata services which requires a CloudStack VR.
>
> Solutions
> (1) create a network with dummy(e.g. private) IPv4 addresses. OR
> (2) Create a network offering without any services, then create a shared
> network with the new offering.
>
> -Wei
>
> On Mon, 28 Mar 2022 at 11:23, Ranjit Jadhav 
> wrote:
>
> > Hello,
> >
> > I am getting the following error while trying to create a shared network
> > for IPv6, recently upgraded to 4.16.1
> >
> > "Currently IPv6-only Shared network with Virtual Router provider is not
> > supported."
> >
> > Any guideline will be of great help.
> >
> > Thanks and regards,
> > Ranjit
> >
>

Re: IPv6 error :: ACS 4.16.1

[Wido den Hollander]

Op 28-03-2022 om 11:31 schreef Wei ZHOU:

Hi Ranjit,

Currently CloudStack virtual routers must configure an IPv4 address on each
interface. Therefore you cannot create shared networks with only IPv6.
However, the default network offering for shared networks supports
dhcp/dns/userdata services which requires a CloudStack VR.



This is something we need to fix though. We probably want to have a 
'default' Link-Local (fe80::) IPv6 address where the VR also listens on 
where we can serve userdata from.


This way the VR can also work in a IPv6-only environment and distribute 
the userdata.


Just food for thought, no real plans from my side :-)

Wido


Solutions
(1) create a network with dummy(e.g. private) IPv4 addresses. OR
(2) Create a network offering without any services, then create a shared
network with the new offering.

-Wei

On Mon, 28 Mar 2022 at 11:23, Ranjit Jadhav 
wrote:


Hello,

I am getting the following error while trying to create a shared network
for IPv6, recently upgraded to 4.16.1

"Currently IPv6-only Shared network with Virtual Router provider is not
supported."

Any guideline will be of great help.

Thanks and regards,
Ranjit

Re: IPv6 error :: ACS 4.16.1

[Wei ZHOU]

Hi Ranjit,

Currently CloudStack virtual routers must configure an IPv4 address on each
interface. Therefore you cannot create shared networks with only IPv6.
However, the default network offering for shared networks supports
dhcp/dns/userdata services which requires a CloudStack VR.

Solutions
(1) create a network with dummy(e.g. private) IPv4 addresses. OR
(2) Create a network offering without any services, then create a shared
network with the new offering.

-Wei

On Mon, 28 Mar 2022 at 11:23, Ranjit Jadhav 
wrote:

> Hello,
>
> I am getting the following error while trying to create a shared network
> for IPv6, recently upgraded to 4.16.1
>
> "Currently IPv6-only Shared network with Virtual Router provider is not
> supported."
>
> Any guideline will be of great help.
>
> Thanks and regards,
> Ranjit
>

Re: IPv6 dhcp problem

[Wido den Hollander]

Op 11-08-2021 om 01:00 schreef Mustafa KALIR:

Hello there,

I installed Cloudstack System with Advanced Zone. And I defined IPv6. However, 
the centos 7 template I prepared does not pull the ipv6 information. How can i 
solve this problem. The below document does not work.



Have you made sure your routers are sending out IPv6 Router 
Advertisements where they advertise the correct IPv6 /64 prefix?


Wido


https://docs.cloudstack.apache.org/en/latest/plugins/ipv6.html

RE: IPV6 in Isolated/VPC networks

[Alex Mattioli]

Hi Kristaps,

The operator will need to add two ranges to ACS.

1) /64 for the physical network (just as we do with public IPV4 at the moment), 
this IPs will be used for the outside interface of the VRs
2) 1 or more /64 or larger (ideally /56s I'd say), which ACS will subnet into 
/64s and assign them to the inside interfaces of the VRs.

Cheers
Alex

 


-Original Message-
From: Kristaps Cudars  
Sent: 12 August 2021 10:37
To: d...@cloudstack.apache.org
Cc: users@cloudstack.apache.org
Subject: Re: IPV6 in Isolated/VPC networks

Hi Rohit,

I think its excellent and satisfies all involved sides.

We will be in front seats bringing this to production.


Suggestion:

This

*   At the zone level root-admin specifies a /64 public range that will be
used for VRs, then

they can add a /48, or /56 IPv6 range for guest networks (to be used by 
isolated networks and VPC tiers)


To

*   At the zone level root-admin specifies a /64 public range that will be
used for VRs, then

they can add one or more ranges equal or larger than /64 IPv6 and ACS will 
subnet it in /64s

for Isolated networks or VPC tiers.


Way:

RIPE NCC in case of PI (Provider Independent) range is granting /48 and its 
important to have

possibility of assigning several subnets, for example /58 to be used for 
internal /64 allocations.

If you want to extend your /48 you will need to have good explanation why you 
want it.
RIPE NCC: Organisations requesting an IPv6 PI assignment larger than a /48 need 
to provide

additional documentation to justify the larger assignment size.

The RIPE NCC will evaluate the PI assignment request and assign an independent 
IPv6 prefix

to the End User organization directly.

On Wed, 11 Aug 2021 at 15:26, Rohit Yadav  wrote:

> Hi all,
>
> Thanks for your feedback and ideas, I've gone ahead with discussing 
> them with Alex and came up with a PoC/design which can be implemented 
> in the following phases:
>
>   *   Phase1: implement ipv6 support in isolated networks and VPC with
> static routing
>   *   Phase2: discuss and implement support for dynamic routing (TBD)
>
> For Phase1 here's the high-level proposal:
>
>   *   IPv6 address management:
>  *   At the zone level root-admin specifies a /64 public range that
> will be used for VRs, then they can add a /48, or /56 IPv6 range for 
> guest networks (to be used by isolated networks and VPC tiers)
>  *   On creation of any IPv6 enabled isolated network or VPC tier,
> from the /48 or /56 block a /64 network is allocated/used
>  *   We assume SLAAC and autoconfiguration, no DHCPv6 in the zone
> (discuss: is privacy a concern, can privacy extensions rfc4941 of 
> slaac be
> explored?)
>   *   Network offerings: root-admin can create new network offerings (with
> VPC too) that specifies a network stack option:
>  *   ipv4 only (default, for backward compatibility all
> networks/offerings post-upgrade migrate to this option)
>  *   ipv4-and-ipv6
>  *   ipv6-only (this can be phase 1.b)
>  *   A new routing option: static (phase1), dynamic (phase2, with
> multiple sub-options such as ospf/bgp etc...)
>   *   VR changes:
>  *   VR gets its guest and public nics set to inet6 auto
>  *   For each /64 allocated to guest network and VPC tiers, radvd is
> configured to do RA
>  *   Firewall: a new ipv6 zone/chain is created for ipv6 where ipv6
> firewall rules (ACLs, ingress, egress) are implemented; ACLs between 
> VPC tiers are managed/implemented by ipv6 firewall on VR
>  *   It is assumed that static routes are created on the core/main
> router by the admin or automated using some scripts/tools; for this 
> CloudStack will announce events with details of /64 networks and VR's 
> public IPv6 address that can be consumed by a rabbitmq/message bus 
> client (for example), or a custom cron job or script as part of orchestration.
> (this wouldn't be necessary for dynamic routing bgp with phase2)
>   *   Guest Networking: With SLAAC, it's easy for CloudStack to calculate
> allocate and use a /64 and determine the IPv6 address of VR nics and 
> guest VM nics
>  *   A user create an isolated network/VPC with an offering that is
> ipv6 enabled
>  *   A user can manage firewall for the IPv6 address/guest nics;
> there'll be no port forward and LB feature though for IPv6
>  *   A users can run workloads in the guest VMs that listen on
> publically routable ipv6 addresses
>  *   Usage/billing etc continue to work, no change needed
>
> Network layout:
>
> [core/ISP router] -> [VR] -> [guest netwokr or VPC tier on a VLAN] -> 
> [guest VMs/nics] *core/ISP router needs static routes to be added 
> (manually or automated), assumes a /48 or /56 configured for the zone
>
> Thoughts, feedback?
>
> Proof-of-concept commentary: h

Re: IPV6 in Isolated/VPC networks

[Kristaps Cudars]

>
> I create a LAN ipv6 (public network for CloudStack VR): at subnet/prefix 0:
> LAN IPv6 address: 2001:470:ed36:0::1/64
> Address mode: SLAAC+stateless DHCP (no dhcpv6)
>   *
>   *
> In the isolated VR, I enabled ipv6 as:
> net.ipv6.conf.all.disable_ipv6 = 0
> net.ipv6.conf.all.forwarding = 1
> net.ipv6.conf.all.accept_ra = 1
> net.ipv6.conf.all.accept_redirects = 1
> net.ipv6.conf.all.autoconf = 1
>
> Set up a IPv6 nameserver/dns in /etc/resolve.conf
> And configured the nics:
> echo iface eth0 inet6 auto >> /etc/network/interfaces
> echo iface eth2 inet6 auto >> /etc/network/interfaces
> /etc/init.d/networking restart
> Next, restart ACS isolated network without cleanup to have it reconfigure
> IPv4 nics, firewall, NAT etc
>
>   *
> Next, I created a /64 network for the isolated guest network on eth0 of VR
> using radvd:
>
> # cat /etc/radvd.conf
> interface eth0
> {
> AdvSendAdvert on;
> MinRtrAdvInterval 5;
> MaxRtrAdvInterval 15;
> prefix 2001:470:ed36:1::/64
> {
> AdvOnLink on;
> AdvAutonomous on;
> };
> };
> systemctl restart radvd
> All guest VMs nics and VR's eth0 gets IPv6 address (SLAAC) in this
> ...:1::/64 network
>   *   Finally I added a static route in toy core-router for the new /64
> IPv6 range in the isolated network
> 2001:470:ed36:1::/64 via  dev
> 
>   *
> ... and I enabled firewall rules to allow any traffic to pass for the new
> /64 network
>
> And voila all done! I create a domain  record that points to my guest
> VM IPv6 address a test webserver on
> http://ipv6-isolated-ntwk-demo.yadav.cloud/
>
> (Note: I'll get rid of the tunnel and request a new /48 block after a few
> days, sharing this solely for testing purposes)
>
>
> Regards.
>
> 
> From: Wido den Hollander 
> Sent: Tuesday, July 20, 2021 12:46
> To: d...@cloudstack.apache.org 
> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> Op 19-07-2021 om 20:38 schreef Kristaps Cudars:
> > Hi Wido,
> >
> > I assume that flouting ip will not work grate with ingress/egress acl on
> VR.
> >
> >  From regular ACS user perspective:
> > I have Instance with dualstack its running web app on 443.
> > I want to swap instances for whatever reason.
> > In case of IPv4 change d-nat rule.
> > In case of IPv6 if flouting IP was not created upfront he will need to
> change dns entry that usually has 24h ttl. Inconvenience degradation in
> experience.
> >
>
> Yes, but, keep in mind that the IP you are using can also be terminated
> on the VR where HAProxy proxies request to the backend VM (could even be
> v4!)
>
> I'm not against DHCPv6, but I have seen many issues with implementing
> it. Therefor I always stick to SLAAC.
>
> >  From ACS admin perspective:
> > I don’t want to have these tickets in helpdesk.
> > You needed to create another flouting IP that it would be seamless- will
> not work as answer.
> >
>
> I understand that as well.
>
> Wido
>
> >
> > On 2021/07/19 09:05:54, Wido den Hollander  wrote:
> >>
> >>
> >> Op 16-07-2021 om 21:46 schreef Kristaps Cudars:
> >>> Hi Wido,
> >>>
> >>> Your proposal is to sacrifice ability to reassign IPv6 to instance,
> have internal domain prefix, and list/db in ACS what IPv6 has been assigned
> to what instance and go with RA and SLAAC. For route signaling to switch
> use BGP/OSPFv3 or manual pre-creation.
> >>>
> >>
> >> You can still list the IPs which have been assigned. You'll know exactly
> >> what IPv6 address a VM has because of the prefix + MAC. Privacy
> >> Extensions need to be disabled in the VM.
> >>
> >> This already works in CloudStack in Shared Networks in this way.
> >>
> >> Using secondary IPs you can always have 'floating' IPv6 addressess.
> >>
> >> Wido
> >>
> >>> Option with RA and managed flag that DHCPv6 is in use to support
> preset information and ability to create route information from ACS is not
> an option as DHCPv6 its failing?
> >>>
> >>>
> >>> On 2021/07/16 15:17:42, Wido den Hollander  wrote:
> >>>>
> >>>>
> >>>> Op 16-07-2021 om 16:42 schreef Hean Seng:
> >>>>> Hi Wido,
> >>>>>
> >>>>> In current setup,  each Cloudstack have own VR, so in this new  IPv6
> subnet
> >>>>> allocation , each VR (which have Frr) will need to have peering with
> I

Re: IPV6 in Isolated/VPC networks

[Hean Seng]

p to have it reconfigure
> IPv4 nics, firewall, NAT etc
>
>   *
> Next, I created a /64 network for the isolated guest network on eth0 of VR
> using radvd:
>
> # cat /etc/radvd.conf
> interface eth0
> {
> AdvSendAdvert on;
> MinRtrAdvInterval 5;
> MaxRtrAdvInterval 15;
> prefix 2001:470:ed36:1::/64
> {
> AdvOnLink on;
> AdvAutonomous on;
> };
> };
> systemctl restart radvd
> All guest VMs nics and VR's eth0 gets IPv6 address (SLAAC) in this
> ...:1::/64 network
>   *   Finally I added a static route in toy core-router for the new /64
> IPv6 range in the isolated network
> 2001:470:ed36:1::/64 via  dev
> 
>   *
> ... and I enabled firewall rules to allow any traffic to pass for the new
> /64 network
>
> And voila all done! I create a domain  record that points to my guest
> VM IPv6 address a test webserver on
> http://ipv6-isolated-ntwk-demo.yadav.cloud/
>
> (Note: I'll get rid of the tunnel and request a new /48 block after a few
> days, sharing this solely for testing purposes)
>
>
> Regards.
>
> 
> From: Wido den Hollander 
> Sent: Tuesday, July 20, 2021 12:46
> To: d...@cloudstack.apache.org 
> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> Op 19-07-2021 om 20:38 schreef Kristaps Cudars:
> > Hi Wido,
> >
> > I assume that flouting ip will not work grate with ingress/egress acl on
> VR.
> >
> >  From regular ACS user perspective:
> > I have Instance with dualstack its running web app on 443.
> > I want to swap instances for whatever reason.
> > In case of IPv4 change d-nat rule.
> > In case of IPv6 if flouting IP was not created upfront he will need to
> change dns entry that usually has 24h ttl. Inconvenience degradation in
> experience.
> >
>
> Yes, but, keep in mind that the IP you are using can also be terminated
> on the VR where HAProxy proxies request to the backend VM (could even be
> v4!)
>
> I'm not against DHCPv6, but I have seen many issues with implementing
> it. Therefor I always stick to SLAAC.
>
> >  From ACS admin perspective:
> > I don’t want to have these tickets in helpdesk.
> > You needed to create another flouting IP that it would be seamless- will
> not work as answer.
> >
>
> I understand that as well.
>
> Wido
>
> >
> > On 2021/07/19 09:05:54, Wido den Hollander  wrote:
> >>
> >>
> >> Op 16-07-2021 om 21:46 schreef Kristaps Cudars:
> >>> Hi Wido,
> >>>
> >>> Your proposal is to sacrifice ability to reassign IPv6 to instance,
> have internal domain prefix, and list/db in ACS what IPv6 has been assigned
> to what instance and go with RA and SLAAC. For route signaling to switch
> use BGP/OSPFv3 or manual pre-creation.
> >>>
> >>
> >> You can still list the IPs which have been assigned. You'll know exactly
> >> what IPv6 address a VM has because of the prefix + MAC. Privacy
> >> Extensions need to be disabled in the VM.
> >>
> >> This already works in CloudStack in Shared Networks in this way.
> >>
> >> Using secondary IPs you can always have 'floating' IPv6 addressess.
> >>
> >> Wido
> >>
> >>> Option with RA and managed flag that DHCPv6 is in use to support
> preset information and ability to create route information from ACS is not
> an option as DHCPv6 its failing?
> >>>
> >>>
> >>> On 2021/07/16 15:17:42, Wido den Hollander  wrote:
> >>>>
> >>>>
> >>>> Op 16-07-2021 om 16:42 schreef Hean Seng:
> >>>>> Hi Wido,
> >>>>>
> >>>>> In current setup,  each Cloudstack have own VR, so in this new  IPv6
> subnet
> >>>>> allocation , each VR (which have Frr) will need to have peering with
> ISP
> >>>>> router (and either BGP or Static Route) , and there is 1000
> Acocunts,  it
> >>>>> will 1000 BGP session with ISP router ,  Am I right for this ? or I
> >>>>> understand wrong .
> >>>>>
> >>>>
> >>>> Yes, that is correct. A /56 would also be sufficient or a /60 which is
> >>>> enough to allocate a few /64 subnets.
> >>>>
> >>>> 1000 BGP connections isn't really a problem for a proper router at the
> >>>> ISP. OSPF(v3) would be better, but as I said that's poorly supported.
> >>>>
> >>>> The ISP could also install 1000 static routes, but that means that the
> >>>&g

Re: IPV6 in Isolated/VPC networks

[Rohit Yadav]

and request a new /48 block after a few days, 
sharing this solely for testing purposes)


Regards.


From: Wido den Hollander 
Sent: Tuesday, July 20, 2021 12:46
To: d...@cloudstack.apache.org 
Subject: Re: IPV6 in Isolated/VPC networks



Op 19-07-2021 om 20:38 schreef Kristaps Cudars:
> Hi Wido,
>
> I assume that flouting ip will not work grate with ingress/egress acl on VR.
>
>  From regular ACS user perspective:
> I have Instance with dualstack its running web app on 443.
> I want to swap instances for whatever reason.
> In case of IPv4 change d-nat rule.
> In case of IPv6 if flouting IP was not created upfront he will need to change 
> dns entry that usually has 24h ttl. Inconvenience degradation in experience.
>

Yes, but, keep in mind that the IP you are using can also be terminated
on the VR where HAProxy proxies request to the backend VM (could even be
v4!)

I'm not against DHCPv6, but I have seen many issues with implementing
it. Therefor I always stick to SLAAC.

>  From ACS admin perspective:
> I don’t want to have these tickets in helpdesk.
> You needed to create another flouting IP that it would be seamless- will not 
> work as answer.
>

I understand that as well.

Wido

>
> On 2021/07/19 09:05:54, Wido den Hollander  wrote:
>>
>>
>> Op 16-07-2021 om 21:46 schreef Kristaps Cudars:
>>> Hi Wido,
>>>
>>> Your proposal is to sacrifice ability to reassign IPv6 to instance, have 
>>> internal domain prefix, and list/db in ACS what IPv6 has been assigned to 
>>> what instance and go with RA and SLAAC. For route signaling to switch use 
>>> BGP/OSPFv3 or manual pre-creation.
>>>
>>
>> You can still list the IPs which have been assigned. You'll know exactly
>> what IPv6 address a VM has because of the prefix + MAC. Privacy
>> Extensions need to be disabled in the VM.
>>
>> This already works in CloudStack in Shared Networks in this way.
>>
>> Using secondary IPs you can always have 'floating' IPv6 addressess.
>>
>> Wido
>>
>>> Option with RA and managed flag that DHCPv6 is in use to support preset 
>>> information and ability to create route information from ACS is not an 
>>> option as DHCPv6 its failing?
>>>
>>>
>>> On 2021/07/16 15:17:42, Wido den Hollander  wrote:
>>>>
>>>>
>>>> Op 16-07-2021 om 16:42 schreef Hean Seng:
>>>>> Hi Wido,
>>>>>
>>>>> In current setup,  each Cloudstack have own VR, so in this new  IPv6 
>>>>> subnet
>>>>> allocation , each VR (which have Frr) will need to have peering with ISP
>>>>> router (and either BGP or Static Route) , and there is 1000 Acocunts,  it
>>>>> will 1000 BGP session with ISP router ,  Am I right for this ? or I
>>>>> understand wrong .
>>>>>
>>>>
>>>> Yes, that is correct. A /56 would also be sufficient or a /60 which is
>>>> enough to allocate a few /64 subnets.
>>>>
>>>> 1000 BGP connections isn't really a problem for a proper router at the
>>>> ISP. OSPF(v3) would be better, but as I said that's poorly supported.
>>>>
>>>> The ISP could also install 1000 static routes, but that means that the
>>>> ISP's router needs to have those configured.
>>>>
>>>> http://docs.frrouting.org/en/latest/ospf6d.html
>>>> (While looking up this URL I see that Frr recently put in a lot of work
>>>> in OSPFv3, seems better now)
>>>>
>>>>> I understand IPv6 is different then IPv4, and in IPv6 it suppose each
>>>>> devices have own IP. It just how to realize in easy way.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
 

> On Fri, Jul 16, 2021 at 8:17 PM Wido den Hollander  wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> Op 16-07-2021 om 05:54 schreef Hean Seng:
>>>>>>> Hi Wido,
>>>>>>>
>>>>>>> My initial thought is not like this,  it is the /48 at ISP router, and
>>>>>> /64
>>>>>>> subnet assign to AdvanceZoneVR,   AdvanceZoneVR responsible is
>>>>>>> distribule IPv6 ip (from the assigned /64 sunet) to VM,  and not routing
>>>>>>> the traffic,   in the VM that get the IPv6 IP will default route to ISP
>>>>>>> router as gw.   It can may be a bridge over 

Re: IPV6 in Isolated/VPC networks

[Wido den Hollander]

;.  I see two options:
> 1) Private AS number on a per-zone basis
> 2) Root Admin assigned AS number on a domain/account

basis

> 3) End-user driven AS number on a per network basis

(for

   bring your
> own AS and IP scenario)
>
> Thoughts?
>
> Cheers
> Alex
>
>
>
>
> -Original Message-
> From: Wido den Hollander mailto:w...@widodh.nl> <mailto:w...@widodh.nl
   <mailto:w...@widodh.nl>>>
> Sent: 13 July 2021 15:08
> To: d...@cloudstack.apache.org
   <mailto:d...@cloudstack.apache.org>
   <mailto:d...@cloudstack.apache.org
   <mailto:d...@cloudstack.apache.org>>;
> Alex Mattioli mailto:alex.matti...@shapeblue.com>
> <mailto:alex.matti...@shapeblue.com
   <mailto:alex.matti...@shapeblue.com>>>
> Cc: Wei Zhou mailto:wei.z...@shapeblue.com>
> <mailto:wei.z...@shapeblue.com
   <mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
> mailto:rohit.ya...@shapeblue.com>
   <mailto:rohit.ya...@shapeblue.com
   <mailto:rohit.ya...@shapeblue.com>>>;
> Gabriel Beims Bräscher mailto:gabr...@pcextreme.nl>
> <mailto:gabr...@pcextreme.nl 
gabr...@pcextreme.nl



> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> On 7/7/21 1:16 PM, Alex Mattioli wrote:
>  > Hi all,
>  > @Wei Zhou<mailto:wei.z...@shapeblue.com
   <mailto:wei.z...@shapeblue.com>
> <mailto:wei.z...@shapeblue.com
   <mailto:wei.z...@shapeblue.com>>> @Rohit
> Yadav<mailto:rohit.ya...@shapeblue.com
   <mailto:rohit.ya...@shapeblue.com>
> <mailto:rohit.ya...@shapeblue.com
   <mailto:rohit.ya...@shapeblue.com>>> and myself are
   investigating how
> to enable IPV6 support on Isolated and VPC networks

and

   would like
> your input on it.
>  > At the moment we are looking at implementing FRR

with

   BGP (and
> possibly OSPF) on the ACS VR.
>  >
>  > We are looking for requirements, recommendations,
   ideas, rants,
> etc...etc...
>  >
>
> Ok! Here we go.
>
> I think that you mean that the VR will actually

route

the

   IPv6
> traffic and for that you need to have a way of

getting

a

   subnet
> routed to the VR.
>
> BGP is probably you best bet here. Although OSPFv3
   technically
> supports this it is very badly implemented in Frr

for

   example.
>
> Now FRR is a very good router and one of the fancy
   features it
> supports is BGP Unnumered. This allows for auto
   configuration of BGP
> over a L2 network when both sides are sending Router
   Advertisements.
> This is very easy for flexible BGP configurations

where

   both sides
> have dynamic IPs.
>
> What you want to do is that you get a /56, /48 or
   something which is
>  >/64 bits routed to the VR.
>
> Now you can sub-segment this into separate /64

subnets.

   You don't
> want to go smaller then a /64 is that prevents you

from

   using SLAAC
> for IPv6 address configuration. This is how it works

for

   Shared
> Networks now in Basic and Advanced Zones.
>
> FRR can now also send out the Router Advertisements

on

   the downlinks
> sending out:
>
> - DNS servers
> - DNS domain
> - Prefix (/64) to be used
>
> There is no need for DHCPv6. You can calculate the

IPv6

   address the
> VM will obtain by using the MAC and the prefix.
>
> So in short:
>
> - Using BGP you routed a /48 to the VR
> - Now you split this into /64 subnets towards the
   isolated networks
>
> Wido
>
>  > Alex Mattioli
>  >
>  >
>  >
>  >
>
>
>
> --
> Regards,
> Hean Seng



   --
   Regards,
   Hean Seng



--
Regards,
Hean Seng

Re: IPV6 in Isolated/VPC networks

[Hean Seng]

IPv6 2000::200:201::1
> >>>>>>
> >>>>>> So in cloudstack need to allow  user to enter ,  IPv6
> >>>>>   gwateway , and
> >>>>>> the  /48 Ipv6 prefix , then it will self allocate the
> /64
> >> ip
> >>>>>   to the VR ,
> >>>>>> and maintain make sure not ovelap allocation
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>   But NAT is truly not the solution with IPv6. IPv6 is
> supposed
> >> to
> >>>> be
> >>>>>   routable. In addition you should avoid DHCPv6 as much as
> >>>>>   possible as
> >>>>>   that's not really the intended use-case for address
> allocation
> >>>>>   with IPv6.
> >>>>>
> >>>>>   In order to route an /48 IPv6 subnet to the VR you have a
> few
> >>>>>   possibilities:
> >>>>>
> >>>>>   - Static route from the upperlying routers which are
> outside
> >> of
> >>>>>   CloudStack
> >>>>>   - BGP
> >>>>>   - OSPFv3 (broken in most cases!)
> >>>>>   - DHCPv6 Prefix Delegation
> >>>>>
> >>>>>   BGP and/or Static routes are still the best bet here.
> >>>>>
> >>>>>   So what you do is that you tell CloudStack that you will
> route
> >>>>>   2001:db8::/48 to the VR, the VR can then use that to split
> it
> >> up
> >>>>>   into
> >>>>>   multiple /64 subnets going towards the instances:
> >>>>>
> >>>>>   - 2001:db8::/64
> >>>>>   - 2001:db8:1::/64
> >>>>>   - 2001:db8:2::/64
> >>>>>   ...
> >>>>>   - 2001:db8:f::/64
> >>>>>
> >>>>>   And go on.
> >>>>>
> >>>>>   In case of BGP you indeed have to tell the VR a few things:
> >>>>>
> >>>>>   - It's own AS number
> >>>>>   - The peer's address(es)
> >>>>>
> >>>>>   With FRR you can simply say:
> >>>>>
> >>>>>   neighbor 2001:db8:4fa::179 remote-as external
> >>>>>
> >>>>>   The /48 you need to have at the VR anyway in case of
> either a
> >>>>>   static
> >>>>>   route or BGP.
> >>>>>
> >>>>>   We just need to add a NullRoute on the VR for that /48 so
> that
> >>>>>   traffic
> >>>>>   will not be routed to the upper gateway in case of the VR
> >> can't
> >>>>>   find a
> >>>>>   route.
> >>>>>
> >>>>>   Wido
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
> >>>>>>  >>>>>   <mailto:alex.matti...@shapeblue.com>
> >>>>>   <mailto:alex.matti...@shapeblue.com
> >>>>>   <mailto:alex.matti...@shapeblue.com>>> wrote:
> >>>>>>
> >>>>>> Hi Wido,
> >>>>>> That's pretty much in line with our thoughts, thanks
> >> for
> >>>>>   the input.
> >>>>>> I believe we agree on the following points then:
> >>>>>>
> >>>>>> - FRR with BGP (no OSPF)
> >>>>>> - Route /48 (or/56) down to the VR
> >>>>>> - /64 per network
> >>>>>> - SLACC for IP addressing
> >>>>>>
> >>>>>> I believe the next big question is then "on which
> level
> >

Re: IPV6 in Isolated/VPC networks

[Wido den Hollander]

   > that would be complicated .
   >
   > We only need assign  IPv6 's /64 prefix to Virtual Router
  (VR) in NAT
   > zone, and the VR responsible to deliver single IPv6 to VM

via

  DHCP6.
   >
   > In VR, you need to have Default IPv6 route to  Physical
  Router's /48. IP
   > as IPv6 Gateway.  Thens should be done .
   >
   > Example :
   > Physical Router Interface
   >   IPv6 IP : 2000:::1/48
   >
   > Cloudstack  virtual router : 2000::200:201::1/64 with
  default ipv6
   > route to router ip 2000:::1
   > and Clodustack Virtual router dhcp allocate IP to VM , and
  VM will have
   > default route to VR. IPv6 2000::200:201::1
   >
   > So in cloudstack need to allow  user to enter ,  IPv6
  gwateway , and
   > the  /48 Ipv6 prefix , then it will self allocate the /64

ip

  to the VR ,
   > and maintain make sure not ovelap allocation
   >
   >

  But NAT is truly not the solution with IPv6. IPv6 is supposed

to

be

  routable. In addition you should avoid DHCPv6 as much as
  possible as
  that's not really the intended use-case for address allocation
  with IPv6.

  In order to route an /48 IPv6 subnet to the VR you have a few
  possibilities:

  - Static route from the upperlying routers which are outside

of

  CloudStack
  - BGP
  - OSPFv3 (broken in most cases!)
  - DHCPv6 Prefix Delegation

  BGP and/or Static routes are still the best bet here.

  So what you do is that you tell CloudStack that you will route
  2001:db8::/48 to the VR, the VR can then use that to split it

up

  into
  multiple /64 subnets going towards the instances:

  - 2001:db8::/64
  - 2001:db8:1::/64
  - 2001:db8:2::/64
  ...
  - 2001:db8:f::/64

  And go on.

  In case of BGP you indeed have to tell the VR a few things:

  - It's own AS number
  - The peer's address(es)

  With FRR you can simply say:

  neighbor 2001:db8:4fa::179 remote-as external

  The /48 you need to have at the VR anyway in case of either a
  static
  route or BGP.

  We just need to add a NullRoute on the VR for that /48 so that
  traffic
  will not be routed to the upper gateway in case of the VR

can't

  find a
  route.

  Wido

   >
   >
   >
   >
   >
   > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
   > mailto:alex.matti...@shapeblue.com>
  <mailto:alex.matti...@shapeblue.com
  <mailto:alex.matti...@shapeblue.com>>> wrote:
   >
   > Hi Wido,
   > That's pretty much in line with our thoughts, thanks

for

  the input.
   > I believe we agree on the following points then:
   >
   > - FRR with BGP (no OSPF)
   > - Route /48 (or/56) down to the VR
   > - /64 per network
   > - SLACC for IP addressing
   >
   > I believe the next big question is then "on which level
  of ACS do we
   > manage AS numbers?".  I see two options:
   > 1) Private AS number on a per-zone basis
   > 2) Root Admin assigned AS number on a domain/account

basis

   > 3) End-user driven AS number on a per network basis

(for

  bring your
   > own AS and IP scenario)
   >
   > Thoughts?
   >
   > Cheers
   > Alex
   >
   >
   >
   >
   > -Original Message-
   > From: Wido den Hollander mailto:w...@widodh.nl> <mailto:w...@widodh.nl
  <mailto:w...@widodh.nl>>>
   > Sent: 13 July 2021 15:08
   > To: d...@cloudstack.apache.org
  <mailto:d...@cloudstack.apache.org>
  <mailto:d...@cloudstack.apache.org
  <mailto:d...@cloudstack.apache.org>>;
   > Alex Mattioli mailto:alex.matti...@shapeblue.com>
   > <mailto:alex.matti...@shapeblue.com
  <mailto:alex.matti...@shapeblue.com>>>
   > Cc: Wei Zhou mailto:wei.z...@shapeblue.com>
   > <mailto:wei.z...@shapeblue.com
  <mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
   >  

Re: IPV6 in Isolated/VPC networks

[Hean Seng]

gt;>>
> >>>  Wido
> >>>
> >>>   >
> >>>   >
> >>>   >
> >>>   >
> >>>   >
> >>>   > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
> >>>   >  >>>  <mailto:alex.matti...@shapeblue.com>
> >>>  <mailto:alex.matti...@shapeblue.com
> >>>  <mailto:alex.matti...@shapeblue.com>>> wrote:
> >>>   >
> >>>   > Hi Wido,
> >>>   > That's pretty much in line with our thoughts, thanks
> for
> >>>  the input.
> >>>   > I believe we agree on the following points then:
> >>>   >
> >>>   > - FRR with BGP (no OSPF)
> >>>   > - Route /48 (or/56) down to the VR
> >>>   > - /64 per network
> >>>   > - SLACC for IP addressing
> >>>   >
> >>>   > I believe the next big question is then "on which level
> >>>  of ACS do we
> >>>   > manage AS numbers?".  I see two options:
> >>>   > 1) Private AS number on a per-zone basis
> >>>   > 2) Root Admin assigned AS number on a domain/account
> basis
> >>>   > 3) End-user driven AS number on a per network basis
> (for
> >>>  bring your
> >>>   > own AS and IP scenario)
> >>>   >
> >>>   > Thoughts?
> >>>   >
> >>>   > Cheers
> >>>   > Alex
> >>>   >
> >>>   >
> >>>   >
> >>>   >
> >>>   > -Original Message-
> >>>   > From: Wido den Hollander  >>>  <mailto:w...@widodh.nl> <mailto:w...@widodh.nl
> >>>  <mailto:w...@widodh.nl>>>
> >>>   > Sent: 13 July 2021 15:08
> >>>   > To: d...@cloudstack.apache.org
> >>>  <mailto:d...@cloudstack.apache.org>
> >>>  <mailto:d...@cloudstack.apache.org
> >>>  <mailto:d...@cloudstack.apache.org>>;
> >>>   > Alex Mattioli  >>>  <mailto:alex.matti...@shapeblue.com>
> >>>   > <mailto:alex.matti...@shapeblue.com
> >>>  <mailto:alex.matti...@shapeblue.com>>>
> >>>   > Cc: Wei Zhou  >>>  <mailto:wei.z...@shapeblue.com>
> >>>   > <mailto:wei.z...@shapeblue.com
> >>>  <mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
> >>>   >  >>>  <mailto:rohit.ya...@shapeblue.com>
> >>>  <mailto:rohit.ya...@shapeblue.com
> >>>  <mailto:rohit.ya...@shapeblue.com>>>;
> >>>   > Gabriel Beims Bräscher  >>>  <mailto:gabr...@pcextreme.nl>
> >>>   > <mailto:gabr...@pcextreme.nl  gabr...@pcextreme.nl
> >>>>>
> >>>   > Subject: Re: IPV6 in Isolated/VPC networks
> >>>   >
> >>>   >
> >>>   >
> >>>   > On 7/7/21 1:16 PM, Alex Mattioli wrote:
> >>>   >  > Hi all,
> >>>   >  > @Wei Zhou<mailto:wei.z...@shapeblue.com
> >>>  <mailto:wei.z...@shapeblue.com>
> >>>   > <mailto:wei.z...@shapeblue.com
> >>>  <mailto:wei.z...@shapeblue.com>>> @Rohit
> >>>   > Yadav<mailto:rohit.ya...@shapeblue.com
> >>>  <mailto:rohit.ya...@shapeblue.com>
> >>>   > <mailto:rohit.ya...@shapeblue.com
> >>>  <mailto:rohit.ya...@shapeblue.com>>> and myself are
> >>>  investigating how
> >>>   > to enable IPV6 support on Isolated and VPC networks and
> >>>  would like
> >>>   > your input on it.
> >>>   >  > At the moment we are looking at implementing FRR
> with
> >>>  BGP (and
> >>>   > possibly OSPF) on the ACS VR.
> >>>   >  >
> >>>   >  > We are looking for requirements, recommendations,
> >>>  ideas, rants,
> >>>   > etc...etc...
> >>>   >  >
> >>>   >
> >>>   > Ok! Here we go.
> >>>   >
> >>>   > I think that you mean that the VR will actually route
> the
> >>>  IPv6
> >>>   > traffic and for that you need to have a way of getting
> a
> >>>  subnet
> >>>   > routed to the VR.
> >>>   >
> >>>   > BGP is probably you best bet here. Although OSPFv3
> >>>  technically
> >>>   > supports this it is very badly implemented in Frr for
> >>>  example.
> >>>   >
> >>>   > Now FRR is a very good router and one of the fancy
> >>>  features it
> >>>   > supports is BGP Unnumered. This allows for auto
> >>>  configuration of BGP
> >>>   > over a L2 network when both sides are sending Router
> >>>  Advertisements.
> >>>   > This is very easy for flexible BGP configurations where
> >>>  both sides
> >>>   > have dynamic IPs.
> >>>   >
> >>>   > What you want to do is that you get a /56, /48 or
> >>>  something which is
> >>>   >  >/64 bits routed to the VR.
> >>>   >
> >>>   > Now you can sub-segment this into separate /64 subnets.
> >>>  You don't
> >>>   > want to go smaller then a /64 is that prevents you from
> >>>  using SLAAC
> >>>   > for IPv6 address configuration. This is how it works
> for
> >>>  Shared
> >>>   > Networks now in Basic and Advanced Zones.
> >>>   >
> >>>   > FRR can now also send out the Router Advertisements on
> >>>  the downlinks
> >>>   > sending out:
> >>>   >
> >>>   > - DNS servers
> >>>   > - DNS domain
> >>>   > - Prefix (/64) to be used
> >>>   >
> >>>   > There is no need for DHCPv6. You can calculate the IPv6
> >>>  address the
> >>>   > VM will obtain by using the MAC and the prefix.
> >>>   >
> >>>   > So in short:
> >>>   >
> >>>   > - Using BGP you routed a /48 to the VR
> >>>   > - Now you split this into /64 subnets towards the
> >>>  isolated networks
> >>>   >
> >>>   > Wido
> >>>   >
> >>>   >  > Alex Mattioli
> >>>   >  >
> >>>   >  >
> >>>   >  >
> >>>   >  >
> >>>   >
> >>>   >
> >>>   >
> >>>   > --
> >>>   > Regards,
> >>>   > Hean Seng
> >>>
> >>>
> >>>
> >>>  --
> >>>  Regards,
> >>>  Hean Seng
> >>>
> >>>
> >>>
> >>> --
> >>> Regards,
> >>> Hean Seng
> >
> >
> >
>


-- 
Regards,
Hean Seng

Re: IPV6 in Isolated/VPC networks

[Wido den Hollander]

AT is truly not the solution with IPv6. IPv6 is supposed to

be

 routable. In addition you should avoid DHCPv6 as much as
 possible as
 that's not really the intended use-case for address allocation
 with IPv6.

 In order to route an /48 IPv6 subnet to the VR you have a few
 possibilities:

 - Static route from the upperlying routers which are outside of
 CloudStack
 - BGP
 - OSPFv3 (broken in most cases!)
 - DHCPv6 Prefix Delegation

 BGP and/or Static routes are still the best bet here.

 So what you do is that you tell CloudStack that you will route
 2001:db8::/48 to the VR, the VR can then use that to split it up
 into
 multiple /64 subnets going towards the instances:

 - 2001:db8::/64
 - 2001:db8:1::/64
 - 2001:db8:2::/64
 ...
 - 2001:db8:f::/64

 And go on.

 In case of BGP you indeed have to tell the VR a few things:

 - It's own AS number
 - The peer's address(es)

 With FRR you can simply say:

 neighbor 2001:db8:4fa::179 remote-as external

 The /48 you need to have at the VR anyway in case of either a
 static
 route or BGP.

 We just need to add a NullRoute on the VR for that /48 so that
 traffic
 will not be routed to the upper gateway in case of the VR can't
 find a
 route.

 Wido

  >
  >
  >
  >
  >
  > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
  > mailto:alex.matti...@shapeblue.com>
 <mailto:alex.matti...@shapeblue.com
 <mailto:alex.matti...@shapeblue.com>>> wrote:
  >
  > Hi Wido,
  > That's pretty much in line with our thoughts, thanks for
 the input.
  > I believe we agree on the following points then:
  >
  > - FRR with BGP (no OSPF)
  > - Route /48 (or/56) down to the VR
  > - /64 per network
  > - SLACC for IP addressing
  >
  > I believe the next big question is then "on which level
 of ACS do we
  > manage AS numbers?".  I see two options:
  > 1) Private AS number on a per-zone basis
  > 2) Root Admin assigned AS number on a domain/account basis
  > 3) End-user driven AS number on a per network basis (for
 bring your
  > own AS and IP scenario)
  >
  > Thoughts?
  >
  > Cheers
  > Alex
  >
  >
  >
  >
  > -Original Message-
  > From: Wido den Hollander mailto:w...@widodh.nl> <mailto:w...@widodh.nl
 <mailto:w...@widodh.nl>>>
  > Sent: 13 July 2021 15:08
  > To: d...@cloudstack.apache.org
 <mailto:d...@cloudstack.apache.org>
 <mailto:d...@cloudstack.apache.org
 <mailto:d...@cloudstack.apache.org>>;
  > Alex Mattioli mailto:alex.matti...@shapeblue.com>
  > <mailto:alex.matti...@shapeblue.com
 <mailto:alex.matti...@shapeblue.com>>>
  > Cc: Wei Zhou mailto:wei.z...@shapeblue.com>
  > <mailto:wei.z...@shapeblue.com
 <mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
  > mailto:rohit.ya...@shapeblue.com>
 <mailto:rohit.ya...@shapeblue.com
 <mailto:rohit.ya...@shapeblue.com>>>;
  > Gabriel Beims Bräscher mailto:gabr...@pcextreme.nl>
  > <mailto:gabr...@pcextreme.nl <mailto:gabr...@pcextreme.nl



  > Subject: Re: IPV6 in Isolated/VPC networks
  >
  >
  >
  > On 7/7/21 1:16 PM, Alex Mattioli wrote:
  >  > Hi all,
  >  > @Wei Zhou<mailto:wei.z...@shapeblue.com
 <mailto:wei.z...@shapeblue.com>
  > <mailto:wei.z...@shapeblue.com
 <mailto:wei.z...@shapeblue.com>>> @Rohit
  > Yadav<mailto:rohit.ya...@shapeblue.com
 <mailto:rohit.ya...@shapeblue.com>
  > <mailto:rohit.ya...@shapeblue.com
 <mailto:rohit.ya...@shapeblue.com>>> and myself are
 investigating how
  > to enable IPV6 support on Isolated and VPC networks and
 would like
  > your input on it.
  >  > At the moment we are looking at implementing FRR with
 BGP (and
  > possibly OSPF) on the ACS VR.
  >  >
  >  > 

Re: IPV6 in Isolated/VPC networks

[Hean Seng]

   > and maintain make sure not ovelap allocation
> >  >
> >  >
> >
> > But NAT is truly not the solution with IPv6. IPv6 is supposed to
> be
> > routable. In addition you should avoid DHCPv6 as much as
> > possible as
> > that's not really the intended use-case for address allocation
> > with IPv6.
> >
> > In order to route an /48 IPv6 subnet to the VR you have a few
> > possibilities:
> >
> > - Static route from the upperlying routers which are outside of
> > CloudStack
> > - BGP
> > - OSPFv3 (broken in most cases!)
> > - DHCPv6 Prefix Delegation
> >
> > BGP and/or Static routes are still the best bet here.
> >
> > So what you do is that you tell CloudStack that you will route
> > 2001:db8::/48 to the VR, the VR can then use that to split it up
> > into
> > multiple /64 subnets going towards the instances:
> >
> > - 2001:db8::/64
> > - 2001:db8:1::/64
> > - 2001:db8:2::/64
> > ...
> > - 2001:db8:f::/64
> >
> > And go on.
> >
> > In case of BGP you indeed have to tell the VR a few things:
> >
> > - It's own AS number
> > - The peer's address(es)
> >
> > With FRR you can simply say:
> >
> > neighbor 2001:db8:4fa::179 remote-as external
> >
> > The /48 you need to have at the VR anyway in case of either a
> > static
> > route or BGP.
> >
> > We just need to add a NullRoute on the VR for that /48 so that
> > traffic
> > will not be routed to the upper gateway in case of the VR can't
> > find a
> > route.
> >
> > Wido
> >
> >  >
> >  >
> >  >
> >  >
> >  >
> >  > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
> >  >  > <mailto:alex.matti...@shapeblue.com>
> > <mailto:alex.matti...@shapeblue.com
> > <mailto:alex.matti...@shapeblue.com>>> wrote:
> >  >
> >  > Hi Wido,
> >  > That's pretty much in line with our thoughts, thanks for
> > the input.
> >  > I believe we agree on the following points then:
> >  >
> >  > - FRR with BGP (no OSPF)
> >  > - Route /48 (or/56) down to the VR
> >  > - /64 per network
> >  > - SLACC for IP addressing
> >  >
> >  > I believe the next big question is then "on which level
> > of ACS do we
> >  > manage AS numbers?".  I see two options:
> >  > 1) Private AS number on a per-zone basis
> >  > 2) Root Admin assigned AS number on a domain/account basis
> >  > 3) End-user driven AS number on a per network basis (for
> > bring your
> >  > own AS and IP scenario)
> >  >
> >  > Thoughts?
> >  >
> >  > Cheers
> >  > Alex
> >  >
> >  >
> >  >
> >  >
> >  > -Original Message-
> >  > From: Wido den Hollander  > <mailto:w...@widodh.nl> <mailto:w...@widodh.nl
> > <mailto:w...@widodh.nl>>>
> >  > Sent: 13 July 2021 15:08
> >  > To: d...@cloudstack.apache.org
> > <mailto:d...@cloudstack.apache.org>
> > <mailto:d...@cloudstack.apache.org
> > <mailto:d...@cloudstack.apache.org>>;
> >  > Alex Mattioli  > <mailto:alex.matti...@shapeblue.com>
> >  > <mailto:alex.matti...@shapeblue.com
> > <mailto:alex.matti...@shapeblue.com>>>
> >  > Cc: Wei Zhou  > <mailto:wei.z...@shapeblue.com>
> >  > <mailto:wei.z...@shapeblue.com
> > <mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
> >  >  > <mailto:rohit.ya...@shapeblue.com>
> > <mailto:rohit.ya...@shapeblue.com
> > <mailto:rohit.ya...@shapeblue.com>>>;
> >  > Gabriel 

Re: IPV6 in Isolated/VPC networks

[Wido den Hollander]

level
of ACS do we
 >     manage AS numbers?".  I see two options:
 >     1) Private AS number on a per-zone basis
 >     2) Root Admin assigned AS number on a domain/account basis
 >     3) End-user driven AS number on a per network basis (for
bring your
 >     own AS and IP scenario)
 >
 >     Thoughts?
 >
 >     Cheers
 >     Alex
 >
 >
 >
 >
 >     -Original Message-
 >     From: Wido den Hollander mailto:w...@widodh.nl> <mailto:w...@widodh.nl
<mailto:w...@widodh.nl>>>
 >     Sent: 13 July 2021 15:08
 >     To: d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>
<mailto:d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>>;
 >     Alex Mattioli mailto:alex.matti...@shapeblue.com>
 >     <mailto:alex.matti...@shapeblue.com
<mailto:alex.matti...@shapeblue.com>>>
 >     Cc: Wei Zhou mailto:wei.z...@shapeblue.com>
 >     <mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
     >     mailto:rohit.ya...@shapeblue.com>
<mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>>>;
 >     Gabriel Beims Bräscher mailto:gabr...@pcextreme.nl>
 >     <mailto:gabr...@pcextreme.nl <mailto:gabr...@pcextreme.nl>>>
 >     Subject: Re: IPV6 in Isolated/VPC networks
 >
 >
 >
 >     On 7/7/21 1:16 PM, Alex Mattioli wrote:
 >      > Hi all,
 >      > @Wei Zhou<mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>
 >     <mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>>> @Rohit
 >     Yadav<mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>
 >     <mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>>> and myself are
investigating how
 >     to enable IPV6 support on Isolated and VPC networks and
would like
 >     your input on it.
 >      > At the moment we are looking at implementing FRR with
BGP (and
 >     possibly OSPF) on the ACS VR.
 >      >
 >      > We are looking for requirements, recommendations,
ideas, rants,
 >     etc...etc...
 >      >
 >
 >     Ok! Here we go.
 >
 >     I think that you mean that the VR will actually route the
IPv6
 >     traffic and for that you need to have a way of getting a
subnet
 >     routed to the VR.
 >
 >     BGP is probably you best bet here. Although OSPFv3
technically
 >     supports this it is very badly implemented in Frr for
example.
 >
 >     Now FRR is a very good router and one of the fancy
features it
 >     supports is BGP Unnumered. This allows for auto
configuration of BGP
 >     over a L2 network when both sides are sending Router
Advertisements.
 >     This is very easy for flexible BGP configurations where
both sides
 >     have dynamic IPs.
 >
 >     What you want to do is that you get a /56, /48 or
something which is
 >      >/64 bits routed to the VR.
 >
 >     Now you can sub-segment this into separate /64 subnets.
You don't
 >     want to go smaller then a /64 is that prevents you from
using SLAAC
 >     for IPv6 address configuration. This is how it works for
Shared
 >     Networks now in Basic and Advanced Zones.
 >
 >     FRR can now also send out the Router Advertisements on
the downlinks
 >     sending out:
 >
 >     - DNS servers
 >     - DNS domain
 >     - Prefix (/64) to be used
 >
 >     There is no need for DHCPv6. You can calculate the IPv6
address the
 >     VM will obtain by using the MAC and the prefix.
 >
 >     So in short:
 >
 >     - Using BGP you routed a /48 to the VR
 >     - Now you split this into /64 subnets towards the
isolated networks
 >
 >     Wido
 >
 >      > Alex Mattioli
 >      >
 >      >
 >      >
 > 

Re: IPV6 in Isolated/VPC networks

[Hean Seng]

Or explain like this :

1) Cloudstack generate list of /64 subnet from /48 that Network admin
assigned to Cloudstack
2) Cloudsack allocated the subnet (that generated from step1) to Virtual
Router, one Virtual Router have one subniet /64
3) Virtual Router allocate single IPv6 (within the range of /64 allocated
to VR)  to VM






On Thu, Jul 15, 2021 at 6:25 PM Hean Seng  wrote:

> Hi Wido,
>
> I think the /48 is at physical router as gateway , and subnet of /64 at VR
> of Cloudstack.   Cloudstack only keep which /48 prefix and vlan information
> of this /48 to be later split the  /64. to VR.
>
> And the instances is getting singe IPv6 of /64  IP.   The VR is getting
> /64.  The default gateway shall goes to /48 of physical router ip .   In
> this case ,does not need any BGP router .
>
>
> Similar concept as IPv4 :
>
> /48 subnet of IPv6 is equivalent to current /24 subnet of IPv4 that
> created in Network.
> and /64  of IPv6 is equivalent to single IP of IPv4 assign to VM.
>
>
>
>
> On Thu, Jul 15, 2021 at 5:31 PM Wido den Hollander  wrote:
>
>>
>>
>> Op 14-07-2021 om 16:44 schreef Hean Seng:
>> > Hi
>> >
>> > I replied in another thread, i think do not need implement BGP or OSPF,
>> > that would be complicated .
>> >
>> > We only need assign  IPv6 's /64 prefix to Virtual Router (VR) in NAT
>> > zone, and the VR responsible to deliver single IPv6 to VM via DHCP6.
>> >
>> > In VR, you need to have Default IPv6 route to  Physical Router's /48.
>> IP
>> > as IPv6 Gateway.  Thens should be done .
>> >
>> > Example :
>> > Physical Router Interface
>> >   IPv6 IP : 2000:::1/48
>> >
>> > Cloudstack  virtual router : 2000::200:201::1/64 with default ipv6
>> > route to router ip 2000:::1
>> > and Clodustack Virtual router dhcp allocate IP to VM , and  VM will
>> have
>> > default route to VR. IPv6 2000::200:201::1
>> >
>> > So in cloudstack need to allow  user to enter ,  IPv6 gwateway , and
>> > the  /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR
>> ,
>> > and maintain make sure not ovelap allocation
>> >
>> >
>>
>> But NAT is truly not the solution with IPv6. IPv6 is supposed to be
>> routable. In addition you should avoid DHCPv6 as much as possible as
>> that's not really the intended use-case for address allocation with IPv6.
>>
>> In order to route an /48 IPv6 subnet to the VR you have a few
>> possibilities:
>>
>> - Static route from the upperlying routers which are outside of CloudStack
>> - BGP
>> - OSPFv3 (broken in most cases!)
>> - DHCPv6 Prefix Delegation
>>
>> BGP and/or Static routes are still the best bet here.
>>
>> So what you do is that you tell CloudStack that you will route
>> 2001:db8::/48 to the VR, the VR can then use that to split it up into
>> multiple /64 subnets going towards the instances:
>>
>> - 2001:db8::/64
>> - 2001:db8:1::/64
>> - 2001:db8:2::/64
>> ...
>> - 2001:db8:f::/64
>>
>> And go on.
>>
>> In case of BGP you indeed have to tell the VR a few things:
>>
>> - It's own AS number
>> - The peer's address(es)
>>
>> With FRR you can simply say:
>>
>> neighbor 2001:db8:4fa::179 remote-as external
>>
>> The /48 you need to have at the VR anyway in case of either a static
>> route or BGP.
>>
>> We just need to add a NullRoute on the VR for that /48 so that traffic
>> will not be routed to the upper gateway in case of the VR can't find a
>> route.
>>
>> Wido
>>
>> >
>> >
>> >
>> >
>> >
>> > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
>> > mailto:alex.matti...@shapeblue.com>>
>> wrote:
>> >
>> > Hi Wido,
>> > That's pretty much in line with our thoughts, thanks for the input.
>> > I believe we agree on the following points then:
>> >
>> > - FRR with BGP (no OSPF)
>> > - Route /48 (or/56) down to the VR
>> > - /64 per network
>> > - SLACC for IP addressing
>> >
>> > I believe the next big question is then "on which level of ACS do we
>> > manage AS numbers?".  I see two options:
>> > 1) Private AS number on a per-zone basis
>> > 2) Root Admin assigned AS number on a domain/account basis
>> > 3) End-user driven AS number on a per network basis (for bring your
>> > own AS and

Re: IPV6 in Isolated/VPC networks

[Hean Seng]

Hi Wido,

I think the /48 is at physical router as gateway , and subnet of /64 at VR
of Cloudstack.   Cloudstack only keep which /48 prefix and vlan information
of this /48 to be later split the  /64. to VR.

And the instances is getting singe IPv6 of /64  IP.   The VR is getting
/64.  The default gateway shall goes to /48 of physical router ip .   In
this case ,does not need any BGP router .


Similar concept as IPv4 :

/48 subnet of IPv6 is equivalent to current /24 subnet of IPv4 that created
in Network.
and /64  of IPv6 is equivalent to single IP of IPv4 assign to VM.




On Thu, Jul 15, 2021 at 5:31 PM Wido den Hollander  wrote:

>
>
> Op 14-07-2021 om 16:44 schreef Hean Seng:
> > Hi
> >
> > I replied in another thread, i think do not need implement BGP or OSPF,
> > that would be complicated .
> >
> > We only need assign  IPv6 's /64 prefix to Virtual Router (VR) in NAT
> > zone, and the VR responsible to deliver single IPv6 to VM via DHCP6.
> >
> > In VR, you need to have Default IPv6 route to  Physical Router's /48. IP
> > as IPv6 Gateway.  Thens should be done .
> >
> > Example :
> > Physical Router Interface
> >   IPv6 IP : 2000:::1/48
> >
> > Cloudstack  virtual router : 2000::200:201::1/64 with default ipv6
> > route to router ip 2000:::1
> > and Clodustack Virtual router dhcp allocate IP to VM , and  VM will have
> > default route to VR. IPv6 2000::200:201::1
> >
> > So in cloudstack need to allow  user to enter ,  IPv6 gwateway , and
> > the  /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR ,
> > and maintain make sure not ovelap allocation
> >
> >
>
> But NAT is truly not the solution with IPv6. IPv6 is supposed to be
> routable. In addition you should avoid DHCPv6 as much as possible as
> that's not really the intended use-case for address allocation with IPv6.
>
> In order to route an /48 IPv6 subnet to the VR you have a few
> possibilities:
>
> - Static route from the upperlying routers which are outside of CloudStack
> - BGP
> - OSPFv3 (broken in most cases!)
> - DHCPv6 Prefix Delegation
>
> BGP and/or Static routes are still the best bet here.
>
> So what you do is that you tell CloudStack that you will route
> 2001:db8::/48 to the VR, the VR can then use that to split it up into
> multiple /64 subnets going towards the instances:
>
> - 2001:db8::/64
> - 2001:db8:1::/64
> - 2001:db8:2::/64
> ...
> - 2001:db8:f::/64
>
> And go on.
>
> In case of BGP you indeed have to tell the VR a few things:
>
> - It's own AS number
> - The peer's address(es)
>
> With FRR you can simply say:
>
> neighbor 2001:db8:4fa::179 remote-as external
>
> The /48 you need to have at the VR anyway in case of either a static
> route or BGP.
>
> We just need to add a NullRoute on the VR for that /48 so that traffic
> will not be routed to the upper gateway in case of the VR can't find a
> route.
>
> Wido
>
> >
> >
> >
> >
> >
> > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
> > mailto:alex.matti...@shapeblue.com>>
> wrote:
> >
> > Hi Wido,
> > That's pretty much in line with our thoughts, thanks for the input.
> > I believe we agree on the following points then:
> >
> > - FRR with BGP (no OSPF)
> > - Route /48 (or/56) down to the VR
> > - /64 per network
> > - SLACC for IP addressing
> >
> > I believe the next big question is then "on which level of ACS do we
> > manage AS numbers?".  I see two options:
> > 1) Private AS number on a per-zone basis
> > 2) Root Admin assigned AS number on a domain/account basis
> > 3) End-user driven AS number on a per network basis (for bring your
> > own AS and IP scenario)
> >
> > Thoughts?
> >
> > Cheers
> > Alex
> >
> >
> >
> >
> > -Original Message-
> > From: Wido den Hollander mailto:w...@widodh.nl>>
> > Sent: 13 July 2021 15:08
> > To: d...@cloudstack.apache.org <mailto:d...@cloudstack.apache.org>;
> > Alex Mattioli  > <mailto:alex.matti...@shapeblue.com>>
> > Cc: Wei Zhou  > <mailto:wei.z...@shapeblue.com>>; Rohit Yadav
> > mailto:rohit.ya...@shapeblue.com>>;
> > Gabriel Beims Bräscher  > <mailto:gabr...@pcextreme.nl>>
> > Subject: Re: IPV6 in Isolated/VPC networks
> >
> >
> >
> > On 7/7/21 1:16 PM, Alex Mattioli wrote:
> >  > Hi all,
> >  > @Wei Zhou<mailto:wei.z...

Re: IPV6 in Isolated/VPC networks

[Wido den Hollander]

Op 14-07-2021 om 16:44 schreef Hean Seng:

Hi

I replied in another thread, i think do not need implement BGP or OSPF, 
that would be complicated .


We only need assign  IPv6 's /64 prefix to Virtual Router (VR) in NAT 
zone, and the VR responsible to deliver single IPv6 to VM via DHCP6.


In VR, you need to have Default IPv6 route to  Physical Router's /48. IP 
as IPv6 Gateway.  Thens should be done .


Example :
Physical Router Interface
  IPv6 IP : 2000:::1/48

Cloudstack  virtual router : 2000::200:201::1/64 with default ipv6 
route to router ip 2000:::1
and Clodustack Virtual router dhcp allocate IP to VM , and  VM will have 
default route to VR. IPv6 2000::200:201::1


So in cloudstack need to allow  user to enter ,  IPv6 gwateway , and 
the  /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR , 
and maintain make sure not ovelap allocation





But NAT is truly not the solution with IPv6. IPv6 is supposed to be 
routable. In addition you should avoid DHCPv6 as much as possible as 
that's not really the intended use-case for address allocation with IPv6.


In order to route an /48 IPv6 subnet to the VR you have a few possibilities:

- Static route from the upperlying routers which are outside of CloudStack
- BGP
- OSPFv3 (broken in most cases!)
- DHCPv6 Prefix Delegation

BGP and/or Static routes are still the best bet here.

So what you do is that you tell CloudStack that you will route 
2001:db8::/48 to the VR, the VR can then use that to split it up into 
multiple /64 subnets going towards the instances:


- 2001:db8::/64
- 2001:db8:1::/64
- 2001:db8:2::/64
...
- 2001:db8:f::/64

And go on.

In case of BGP you indeed have to tell the VR a few things:

- It's own AS number
- The peer's address(es)

With FRR you can simply say:

neighbor 2001:db8:4fa::179 remote-as external

The /48 you need to have at the VR anyway in case of either a static 
route or BGP.


We just need to add a NullRoute on the VR for that /48 so that traffic 
will not be routed to the upper gateway in case of the VR can't find a 
route.


Wido







On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli 
mailto:alex.matti...@shapeblue.com>> wrote:


Hi Wido,
That's pretty much in line with our thoughts, thanks for the input. 
I believe we agree on the following points then:


- FRR with BGP (no OSPF)
- Route /48 (or/56) down to the VR
- /64 per network
- SLACC for IP addressing

I believe the next big question is then "on which level of ACS do we
manage AS numbers?".  I see two options:
1) Private AS number on a per-zone basis
2) Root Admin assigned AS number on a domain/account basis
3) End-user driven AS number on a per network basis (for bring your
own AS and IP scenario)

Thoughts?

Cheers
Alex




-Original Message-
From: Wido den Hollander mailto:w...@widodh.nl>>
Sent: 13 July 2021 15:08
To: d...@cloudstack.apache.org <mailto:d...@cloudstack.apache.org>;
Alex Mattioli mailto:alex.matti...@shapeblue.com>>
Cc: Wei Zhou mailto:wei.z...@shapeblue.com>>; Rohit Yadav
mailto:rohit.ya...@shapeblue.com>>;
Gabriel Beims Bräscher mailto:gabr...@pcextreme.nl>>
Subject: Re: IPV6 in Isolated/VPC networks



On 7/7/21 1:16 PM, Alex Mattioli wrote:
 > Hi all,
 > @Wei Zhou<mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>> @Rohit
Yadav<mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>> and myself are investigating how
to enable IPV6 support on Isolated and VPC networks and would like
your input on it.
 > At the moment we are looking at implementing FRR with BGP (and
possibly OSPF) on the ACS VR.
 >
 > We are looking for requirements, recommendations, ideas, rants,
etc...etc...
 >

Ok! Here we go.

I think that you mean that the VR will actually route the IPv6
traffic and for that you need to have a way of getting a subnet
routed to the VR.

BGP is probably you best bet here. Although OSPFv3 technically
supports this it is very badly implemented in Frr for example.

Now FRR is a very good router and one of the fancy features it
supports is BGP Unnumered. This allows for auto configuration of BGP
over a L2 network when both sides are sending Router Advertisements.
This is very easy for flexible BGP configurations where both sides
have dynamic IPs.

What you want to do is that you get a /56, /48 or something which is
 >/64 bits routed to the VR.

Now you can sub-segment this into separate /64 subnets. You don't
want to go smaller then a /64 is that prevents you from using SLAAC
for IPv6 address configuration. This is how it works for Shared
Networks now in Basic and Advanced Zones.

FRR can now also send out the Router Advertisements on the downlinks
sen

Re: IPV6 in Isolated/VPC networks

[Hean Seng]

Yes, sorry for that, can use NAT 6 also .I mentiioned DHCP6 , and you
can point the gateway to /48 gw, and this does not need any BGP.  Maintain
BGP or OSPF is good, but is a lot more complicated ,

On Wed, Jul 14, 2021 at 10:57 PM Alex Mattioli 
wrote:

> Hi Hean,
> Do you mean using NAT66?  Or did I miss something?
>
> Regards,
> Alex
>
>
>
>
> -Original Message-
> From: Hean Seng 
> Sent: 14 July 2021 16:44
> To: users@cloudstack.apache.org
> Cc: Wido den Hollander ; d...@cloudstack.apache.org; Wei
> Zhou ; Rohit Yadav ;
> Gabriel Beims Bräscher 
> Subject: Re: IPV6 in Isolated/VPC networks
>
> Hi
>
> I replied in another thread, i think do not need implement BGP or OSPF,
> that would be complicated .
>
> We only need assign  IPv6 's /64 prefix to Virtual Router (VR) in NAT
> zone, and the VR responsible to deliver single IPv6 to VM via DHCP6.
>
> In VR, you need to have Default IPv6 route to  Physical Router's /48. IP as
> IPv6 Gateway.  Thens should be done .
>
> Example :
> Physical Router Interface
>  IPv6 IP : 2000:::1/48
>
> Cloudstack  virtual router : 2000::200:201::1/64 with default ipv6
> route to router ip 2000:::1 and Clodustack Virtual router dhcp allocate
> IP to VM , and  VM will have default route to VR. IPv6 2000::200:201::1
>
> So in cloudstack need to allow  user to enter ,  IPv6 gwateway , and the
> /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR , and
> maintain make sure not ovelap allocation
>
>
>
>
>
>
>
> On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli  >
> wrote:
>
> > Hi Wido,
> > That's pretty much in line with our thoughts, thanks for the input.  I
> > believe we agree on the following points then:
> >
> > - FRR with BGP (no OSPF)
> > - Route /48 (or/56) down to the VR
> > - /64 per network
> > - SLACC for IP addressing
> >
> > I believe the next big question is then "on which level of ACS do we
> > manage AS numbers?".  I see two options:
> > 1) Private AS number on a per-zone basis
> > 2) Root Admin assigned AS number on a domain/account basis
> > 3) End-user driven AS number on a per network basis (for bring your
> > own AS and IP scenario)
> >
> > Thoughts?
> >
> > Cheers
> > Alex
> >
> >
> >
> >
> > -Original Message-
> > From: Wido den Hollander 
> > Sent: 13 July 2021 15:08
> > To: d...@cloudstack.apache.org; Alex Mattioli
> > 
> > Cc: Wei Zhou ; Rohit Yadav <
> > rohit.ya...@shapeblue.com>; Gabriel Beims Bräscher
> > 
> > Subject: Re: IPV6 in Isolated/VPC networks
> >
> >
> >
> > On 7/7/21 1:16 PM, Alex Mattioli wrote:
> > > Hi all,
> > > @Wei Zhou<mailto:wei.z...@shapeblue.com> @Rohit Yadav > rohit.ya...@shapeblue.com> and myself are investigating how to enable
> > IPV6 support on Isolated and VPC networks and would like your input on
> it.
> > > At the moment we are looking at implementing FRR with BGP (and
> > > possibly
> > OSPF) on the ACS VR.
> > >
> > > We are looking for requirements, recommendations, ideas, rants,
> > etc...etc...
> > >
> >
> > Ok! Here we go.
> >
> > I think that you mean that the VR will actually route the IPv6 traffic
> > and for that you need to have a way of getting a subnet routed to the VR.
> >
> > BGP is probably you best bet here. Although OSPFv3 technically
> > supports this it is very badly implemented in Frr for example.
> >
> > Now FRR is a very good router and one of the fancy features it
> > supports is BGP Unnumered. This allows for auto configuration of BGP
> > over a L2 network when both sides are sending Router Advertisements.
> > This is very easy for flexible BGP configurations where both sides have
> dynamic IPs.
> >
> > What you want to do is that you get a /56, /48 or something which is
> > >/64 bits routed to the VR.
> >
> > Now you can sub-segment this into separate /64 subnets. You don't want
> > to go smaller then a /64 is that prevents you from using SLAAC for
> > IPv6 address configuration. This is how it works for Shared Networks
> > now in Basic and Advanced Zones.
> >
> > FRR can now also send out the Router Advertisements on the downlinks
> > sending out:
> >
> > - DNS servers
> > - DNS domain
> > - Prefix (/64) to be used
> >
> > There is no need for DHCPv6. You can calculate the IPv6 address the VM
> > will obtain by using the MAC and the prefix.
> >
> > So in short:
> >
> > - Using BGP you routed a /48 to the VR
> > - Now you split this into /64 subnets towards the isolated networks
> >
> > Wido
> >
> > > Alex Mattioli
> > >
> > >
> > >
> > >
> >
> >
>
> --
> Regards,
> Hean Seng
>


-- 
Regards,
Hean Seng

RE: IPV6 in Isolated/VPC networks

[Alex Mattioli]

Hi Hean,
Do you mean using NAT66?  Or did I miss something?

Regards,
Alex

 


-Original Message-
From: Hean Seng  
Sent: 14 July 2021 16:44
To: users@cloudstack.apache.org
Cc: Wido den Hollander ; d...@cloudstack.apache.org; Wei Zhou 
; Rohit Yadav ; Gabriel 
Beims Bräscher 
Subject: Re: IPV6 in Isolated/VPC networks

Hi

I replied in another thread, i think do not need implement BGP or OSPF, that 
would be complicated .

We only need assign  IPv6 's /64 prefix to Virtual Router (VR) in NAT zone, and 
the VR responsible to deliver single IPv6 to VM via DHCP6.

In VR, you need to have Default IPv6 route to  Physical Router's /48. IP as
IPv6 Gateway.  Thens should be done .

Example :
Physical Router Interface
 IPv6 IP : 2000:::1/48

Cloudstack  virtual router : 2000::200:201::1/64 with default ipv6 route to 
router ip 2000:::1 and Clodustack Virtual router dhcp allocate IP to VM , 
and  VM will have default route to VR. IPv6 2000::200:201::1

So in cloudstack need to allow  user to enter ,  IPv6 gwateway , and the
/48 Ipv6 prefix , then it will self allocate the /64 ip to the VR , and 
maintain make sure not ovelap allocation







On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli 
wrote:

> Hi Wido,
> That's pretty much in line with our thoughts, thanks for the input.  I 
> believe we agree on the following points then:
>
> - FRR with BGP (no OSPF)
> - Route /48 (or/56) down to the VR
> - /64 per network
> - SLACC for IP addressing
>
> I believe the next big question is then "on which level of ACS do we 
> manage AS numbers?".  I see two options:
> 1) Private AS number on a per-zone basis
> 2) Root Admin assigned AS number on a domain/account basis
> 3) End-user driven AS number on a per network basis (for bring your 
> own AS and IP scenario)
>
> Thoughts?
>
> Cheers
> Alex
>
>
>
>
> -Original Message-
> From: Wido den Hollander 
> Sent: 13 July 2021 15:08
> To: d...@cloudstack.apache.org; Alex Mattioli 
> 
> Cc: Wei Zhou ; Rohit Yadav < 
> rohit.ya...@shapeblue.com>; Gabriel Beims Bräscher 
> 
> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> On 7/7/21 1:16 PM, Alex Mattioli wrote:
> > Hi all,
> > @Wei Zhou<mailto:wei.z...@shapeblue.com> @Rohit Yadav rohit.ya...@shapeblue.com> and myself are investigating how to enable
> IPV6 support on Isolated and VPC networks and would like your input on it.
> > At the moment we are looking at implementing FRR with BGP (and 
> > possibly
> OSPF) on the ACS VR.
> >
> > We are looking for requirements, recommendations, ideas, rants,
> etc...etc...
> >
>
> Ok! Here we go.
>
> I think that you mean that the VR will actually route the IPv6 traffic 
> and for that you need to have a way of getting a subnet routed to the VR.
>
> BGP is probably you best bet here. Although OSPFv3 technically 
> supports this it is very badly implemented in Frr for example.
>
> Now FRR is a very good router and one of the fancy features it 
> supports is BGP Unnumered. This allows for auto configuration of BGP 
> over a L2 network when both sides are sending Router Advertisements. 
> This is very easy for flexible BGP configurations where both sides have 
> dynamic IPs.
>
> What you want to do is that you get a /56, /48 or something which is
> >/64 bits routed to the VR.
>
> Now you can sub-segment this into separate /64 subnets. You don't want 
> to go smaller then a /64 is that prevents you from using SLAAC for 
> IPv6 address configuration. This is how it works for Shared Networks 
> now in Basic and Advanced Zones.
>
> FRR can now also send out the Router Advertisements on the downlinks 
> sending out:
>
> - DNS servers
> - DNS domain
> - Prefix (/64) to be used
>
> There is no need for DHCPv6. You can calculate the IPv6 address the VM 
> will obtain by using the MAC and the prefix.
>
> So in short:
>
> - Using BGP you routed a /48 to the VR
> - Now you split this into /64 subnets towards the isolated networks
>
> Wido
>
> > Alex Mattioli
> >
> >
> >
> >
>
>

--
Regards,
Hean Seng

Re: IPV6 in Isolated/VPC networks

[Hean Seng]

Hi

I replied in another thread, i think do not need implement BGP or OSPF,
that would be complicated .

We only need assign  IPv6 's /64 prefix to Virtual Router (VR) in NAT zone,
and the VR responsible to deliver single IPv6 to VM via DHCP6.

In VR, you need to have Default IPv6 route to  Physical Router's /48. IP as
IPv6 Gateway.  Thens should be done .

Example :
Physical Router Interface
 IPv6 IP : 2000:::1/48

Cloudstack  virtual router : 2000::200:201::1/64 with default ipv6
route to router ip 2000:::1
and Clodustack Virtual router dhcp allocate IP to VM , and  VM will have
default route to VR. IPv6 2000::200:201::1

So in cloudstack need to allow  user to enter ,  IPv6 gwateway , and the
/48 Ipv6 prefix , then it will self allocate the /64 ip to the VR , and
maintain make sure not ovelap allocation







On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli 
wrote:

> Hi Wido,
> That's pretty much in line with our thoughts, thanks for the input.  I
> believe we agree on the following points then:
>
> - FRR with BGP (no OSPF)
> - Route /48 (or/56) down to the VR
> - /64 per network
> - SLACC for IP addressing
>
> I believe the next big question is then "on which level of ACS do we
> manage AS numbers?".  I see two options:
> 1) Private AS number on a per-zone basis
> 2) Root Admin assigned AS number on a domain/account basis
> 3) End-user driven AS number on a per network basis (for bring your own AS
> and IP scenario)
>
> Thoughts?
>
> Cheers
> Alex
>
>
>
>
> -Original Message-
> From: Wido den Hollander 
> Sent: 13 July 2021 15:08
> To: d...@cloudstack.apache.org; Alex Mattioli 
> Cc: Wei Zhou ; Rohit Yadav <
> rohit.ya...@shapeblue.com>; Gabriel Beims Bräscher 
> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> On 7/7/21 1:16 PM, Alex Mattioli wrote:
> > Hi all,
> > @Wei Zhou<mailto:wei.z...@shapeblue.com> @Rohit Yadav rohit.ya...@shapeblue.com> and myself are investigating how to enable
> IPV6 support on Isolated and VPC networks and would like your input on it.
> > At the moment we are looking at implementing FRR with BGP (and possibly
> OSPF) on the ACS VR.
> >
> > We are looking for requirements, recommendations, ideas, rants,
> etc...etc...
> >
>
> Ok! Here we go.
>
> I think that you mean that the VR will actually route the IPv6 traffic and
> for that you need to have a way of getting a subnet routed to the VR.
>
> BGP is probably you best bet here. Although OSPFv3 technically supports
> this it is very badly implemented in Frr for example.
>
> Now FRR is a very good router and one of the fancy features it supports is
> BGP Unnumered. This allows for auto configuration of BGP over a L2 network
> when both sides are sending Router Advertisements. This is very easy for
> flexible BGP configurations where both sides have dynamic IPs.
>
> What you want to do is that you get a /56, /48 or something which is
> >/64 bits routed to the VR.
>
> Now you can sub-segment this into separate /64 subnets. You don't want to
> go smaller then a /64 is that prevents you from using SLAAC for IPv6
> address configuration. This is how it works for Shared Networks now in
> Basic and Advanced Zones.
>
> FRR can now also send out the Router Advertisements on the downlinks
> sending out:
>
> - DNS servers
> - DNS domain
> - Prefix (/64) to be used
>
> There is no need for DHCPv6. You can calculate the IPv6 address the VM
> will obtain by using the MAC and the prefix.
>
> So in short:
>
> - Using BGP you routed a /48 to the VR
> - Now you split this into /64 subnets towards the isolated networks
>
> Wido
>
> > Alex Mattioli
> >
> >
> >
> >
>
>

-- 
Regards,
Hean Seng

RE: IPV6 in Isolated/VPC networks

[Alex Mattioli]

Hi Wido,
That's pretty much in line with our thoughts, thanks for the input.  I believe 
we agree on the following points then:

- FRR with BGP (no OSPF)
- Route /48 (or/56) down to the VR
- /64 per network
- SLACC for IP addressing

I believe the next big question is then "on which level of ACS do we manage AS 
numbers?".  I see two options:
1) Private AS number on a per-zone basis
2) Root Admin assigned AS number on a domain/account basis
3) End-user driven AS number on a per network basis (for bring your own AS and 
IP scenario)

Thoughts?

Cheers
Alex

 


-Original Message-
From: Wido den Hollander  
Sent: 13 July 2021 15:08
To: d...@cloudstack.apache.org; Alex Mattioli 
Cc: Wei Zhou ; Rohit Yadav ; 
Gabriel Beims Bräscher 
Subject: Re: IPV6 in Isolated/VPC networks



On 7/7/21 1:16 PM, Alex Mattioli wrote:
> Hi all,
> @Wei Zhou<mailto:wei.z...@shapeblue.com> @Rohit 
> Yadav<mailto:rohit.ya...@shapeblue.com> and myself are investigating how to 
> enable IPV6 support on Isolated and VPC networks and would like your input on 
> it.
> At the moment we are looking at implementing FRR with BGP (and possibly OSPF) 
> on the ACS VR.
> 
> We are looking for requirements, recommendations, ideas, rants, etc...etc...
> 

Ok! Here we go.

I think that you mean that the VR will actually route the IPv6 traffic and for 
that you need to have a way of getting a subnet routed to the VR.

BGP is probably you best bet here. Although OSPFv3 technically supports this it 
is very badly implemented in Frr for example.

Now FRR is a very good router and one of the fancy features it supports is BGP 
Unnumered. This allows for auto configuration of BGP over a L2 network when 
both sides are sending Router Advertisements. This is very easy for flexible 
BGP configurations where both sides have dynamic IPs.

What you want to do is that you get a /56, /48 or something which is
>/64 bits routed to the VR.

Now you can sub-segment this into separate /64 subnets. You don't want to go 
smaller then a /64 is that prevents you from using SLAAC for IPv6 address 
configuration. This is how it works for Shared Networks now in Basic and 
Advanced Zones.

FRR can now also send out the Router Advertisements on the downlinks sending 
out:

- DNS servers
- DNS domain
- Prefix (/64) to be used

There is no need for DHCPv6. You can calculate the IPv6 address the VM will 
obtain by using the MAC and the prefix.

So in short:

- Using BGP you routed a /48 to the VR
- Now you split this into /64 subnets towards the isolated networks

Wido

> Alex Mattioli
> 
>  
> 
> 

Re: IPv6 Issue in Cloudstack

[Hean Seng]

Yes, I means changing ipv6.

Adding secondary IP, seems not adding second IPv6 also .

For my case now, the IPv6 ad MAC is not the same also :

MAC:  link/ether 1e:00:0d:00:01:ec brd ff:ff:ff:ff:ff:ff

IPV6;

inet6 x:x:x:x:1c00:dff:fe00:1ec/64 scope global mngtmpaddr dynamic
   valid_lft 2591848sec preferred_lft 604648sec
inet6 fe80::1c00:dff:fe00:1ec/64 scope link


It seems last 6 digit same, others is different.



On Sat, May 1, 2021 at 3:03 PM Wido den Hollander  wrote:

>
>
> On 5/1/21 8:48 AM, Hean Seng wrote:
> > Hi Wido
> >
> > The issue solved .  Need to configure ra in router vlan.   Previously we
> > set  "ipv6 nd ra suppress" ， for other systems to work, after change to
> > Cloudstack, it need to remove this and make it have announcement of IPv6
> to
> > VM.
> >
>
> Yes. The Routers need to send IPv6 Router Advertisements in order to
> have the VM configure itself and know where to send traffic to.
>
> > By the way,  This way of configuring IPv6,  if IPv6 need to change, how
> can
> > we replace this IPv6 ?
> >
>
> I don't understand this question. Do you mean how to change the IPv6
> address of a VM?
>
> If so, that's not possible. You can add secondary IPs, but the primary
> IP is based on the MAC of the VM.
>
> Wido
>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Sat, May 1, 2021 at 2:37 PM Wido den Hollander 
> wrote:
> >
> >> Can you check with tcpdump on the host and sniff the vnetX device of the
> >> VM to see if you ICMPv6 packages reach the VM?
> >>
> >> Security Grouping with IPv6 works with KVM, so it has to be a
> >> configuration issue somewhere.
> >>
> >> Wido
> >>
> >> On 4/30/21 8:59 PM, Hean Seng wrote:
> >>> Hi
> >>>
> >>> I am using 4.15 , hypervisor is ubuntu 18 , KVM , yes, I am on advance
> >> with
> >>> SG
> >>>
> >>> I set the Security Group:
> >>>
> >>> ICMP
> >>> -1 -1 ::/0
> >>>
> >>> But seems still cannot ping the VM.
> >>>
> >>> Or even add in rules for ALL
> >>>
> >>> All .  All   ::/0
> >>>
> >>>
> >>> Seems not able to PING.
> >>>
> >>>
> >>> After configure , this is the rules in ip6tables
> >>>
> >>>
> >>> Chain i-2-10-VM (1 references)
> >>> target prot opt source   destination
> >>> ACCEPT ipv6-icmpanywhere anywhere
> >>> ACCEPT all  anywhere anywhere state NEW
> >>> DROP   all  anywhere anywhere
> >>>
> >>>
> >>>
> >>>
> >>> Chain i-2-10-VM-eg (1 references)
> >>>
> >>> target prot opt source   destination
> >>>
> >>> RETURN all  anywhere anywhere
> >>>
> >>>
> >>> Chain i-2-10-def (2 references)
> >>>
> >>> target prot opt source   destination
> >>>
> >>> ACCEPT all  anywhere anywhere state
> >>> RELATED,ESTABLISHED
> >>>
> >>> ACCEPT ipv6-icmpfe80::/64ip6-allnodes
>  PHYSDEV
> >>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> >>> router-advertisement HL match HL == 255
> >>>
> >>> RETURN ipv6-icmpanywhere ip6-allrouters
>  PHYSDEV
> >>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> >> router-solicitation
> >>> HL match HL == 255
> >>>
> >>> DROP   ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> >> router-advertisement
> >>>
> >>> RETURN ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> >>> neighbour-solicitation HL match HL == 255
> >>>
> >>> ACCEPT ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> >>> neighbour-solicitation HL match HL == 255
> >>>
> >>> RETURN ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> >>> neighbour-advertisement match-set i-2-10-VM-6 src HL match HL == 255
> >>>
> >>> ACCEPT ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> >>> neighbour-advertisement HL match HL == 255
> >>>
> >>> RETURN ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp packet-too-big
> >>> match-set i-2-10-VM-6 src
> >>>
> >>> ACCEPT ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp packet-too-big
> >>>
> >>> RETURN ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> >>> destination-unreachable match-set i-2-10-VM-6 src
> >>>
> >>> ACCEPT ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> >>> destination-unreachable
> >>>
> >>> RETURN ipv6-icmpanywhere anywhere
>  PHYSDEV
> >>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp time-exceeded
> 

Re: IPv6 Issue in Cloudstack

[Wido den Hollander]

On 5/1/21 8:48 AM, Hean Seng wrote:
> Hi Wido
> 
> The issue solved .  Need to configure ra in router vlan.   Previously we
> set  "ipv6 nd ra suppress" ， for other systems to work, after change to
> Cloudstack, it need to remove this and make it have announcement of IPv6 to
> VM.
> 

Yes. The Routers need to send IPv6 Router Advertisements in order to
have the VM configure itself and know where to send traffic to.

> By the way,  This way of configuring IPv6,  if IPv6 need to change, how can
> we replace this IPv6 ?
> 

I don't understand this question. Do you mean how to change the IPv6
address of a VM?

If so, that's not possible. You can add secondary IPs, but the primary
IP is based on the MAC of the VM.

Wido

> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Sat, May 1, 2021 at 2:37 PM Wido den Hollander  wrote:
> 
>> Can you check with tcpdump on the host and sniff the vnetX device of the
>> VM to see if you ICMPv6 packages reach the VM?
>>
>> Security Grouping with IPv6 works with KVM, so it has to be a
>> configuration issue somewhere.
>>
>> Wido
>>
>> On 4/30/21 8:59 PM, Hean Seng wrote:
>>> Hi
>>>
>>> I am using 4.15 , hypervisor is ubuntu 18 , KVM , yes, I am on advance
>> with
>>> SG
>>>
>>> I set the Security Group:
>>>
>>> ICMP
>>> -1 -1 ::/0
>>>
>>> But seems still cannot ping the VM.
>>>
>>> Or even add in rules for ALL
>>>
>>> All .  All   ::/0
>>>
>>>
>>> Seems not able to PING.
>>>
>>>
>>> After configure , this is the rules in ip6tables
>>>
>>>
>>> Chain i-2-10-VM (1 references)
>>> target prot opt source   destination
>>> ACCEPT ipv6-icmpanywhere anywhere
>>> ACCEPT all  anywhere anywhere state NEW
>>> DROP   all  anywhere anywhere
>>>
>>>
>>>
>>>
>>> Chain i-2-10-VM-eg (1 references)
>>>
>>> target prot opt source   destination
>>>
>>> RETURN all  anywhere anywhere
>>>
>>>
>>> Chain i-2-10-def (2 references)
>>>
>>> target prot opt source   destination
>>>
>>> ACCEPT all  anywhere anywhere state
>>> RELATED,ESTABLISHED
>>>
>>> ACCEPT ipv6-icmpfe80::/64ip6-allnodes PHYSDEV
>>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
>>> router-advertisement HL match HL == 255
>>>
>>> RETURN ipv6-icmpanywhere ip6-allrouters   PHYSDEV
>>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
>> router-solicitation
>>> HL match HL == 255
>>>
>>> DROP   ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
>> router-advertisement
>>>
>>> RETURN ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
>>> neighbour-solicitation HL match HL == 255
>>>
>>> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
>>> neighbour-solicitation HL match HL == 255
>>>
>>> RETURN ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
>>> neighbour-advertisement match-set i-2-10-VM-6 src HL match HL == 255
>>>
>>> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
>>> neighbour-advertisement HL match HL == 255
>>>
>>> RETURN ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp packet-too-big
>>> match-set i-2-10-VM-6 src
>>>
>>> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp packet-too-big
>>>
>>> RETURN ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
>>> destination-unreachable match-set i-2-10-VM-6 src
>>>
>>> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
>>> destination-unreachable
>>>
>>> RETURN ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp time-exceeded
>>> match-set i-2-10-VM-6 src
>>>
>>> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp time-exceeded
>>>
>>> RETURN ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp parameter-problem
>>> match-set i-2-10-VM-6 src
>>>
>>> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
>>> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
>> parameter-problem
>>>
>>> RETURN ipv6-icmpanywhere ff02::16 PHYSDEV
>>> match --physdev-in vnet3 

Re: IPv6 Issue in Cloudstack

[Hean Seng]

Hi Wido

The issue solved .  Need to configure ra in router vlan.   Previously we
set  "ipv6 nd ra suppress" ， for other systems to work, after change to
Cloudstack, it need to remove this and make it have announcement of IPv6 to
VM.

By the way,  This way of configuring IPv6,  if IPv6 need to change, how can
we replace this IPv6 ?










On Sat, May 1, 2021 at 2:37 PM Wido den Hollander  wrote:

> Can you check with tcpdump on the host and sniff the vnetX device of the
> VM to see if you ICMPv6 packages reach the VM?
>
> Security Grouping with IPv6 works with KVM, so it has to be a
> configuration issue somewhere.
>
> Wido
>
> On 4/30/21 8:59 PM, Hean Seng wrote:
> > Hi
> >
> > I am using 4.15 , hypervisor is ubuntu 18 , KVM , yes, I am on advance
> with
> > SG
> >
> > I set the Security Group:
> >
> > ICMP
> > -1 -1 ::/0
> >
> > But seems still cannot ping the VM.
> >
> > Or even add in rules for ALL
> >
> > All .  All   ::/0
> >
> >
> > Seems not able to PING.
> >
> >
> > After configure , this is the rules in ip6tables
> >
> >
> > Chain i-2-10-VM (1 references)
> > target prot opt source   destination
> > ACCEPT ipv6-icmpanywhere anywhere
> > ACCEPT all  anywhere anywhere state NEW
> > DROP   all  anywhere anywhere
> >
> >
> >
> >
> > Chain i-2-10-VM-eg (1 references)
> >
> > target prot opt source   destination
> >
> > RETURN all  anywhere anywhere
> >
> >
> > Chain i-2-10-def (2 references)
> >
> > target prot opt source   destination
> >
> > ACCEPT all  anywhere anywhere state
> > RELATED,ESTABLISHED
> >
> > ACCEPT ipv6-icmpfe80::/64ip6-allnodes PHYSDEV
> > match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> > router-advertisement HL match HL == 255
> >
> > RETURN ipv6-icmpanywhere ip6-allrouters   PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> router-solicitation
> > HL match HL == 255
> >
> > DROP   ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> router-advertisement
> >
> > RETURN ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> > neighbour-solicitation HL match HL == 255
> >
> > ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> > neighbour-solicitation HL match HL == 255
> >
> > RETURN ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> > neighbour-advertisement match-set i-2-10-VM-6 src HL match HL == 255
> >
> > ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> > neighbour-advertisement HL match HL == 255
> >
> > RETURN ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp packet-too-big
> > match-set i-2-10-VM-6 src
> >
> > ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp packet-too-big
> >
> > RETURN ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> > destination-unreachable match-set i-2-10-VM-6 src
> >
> > ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> > destination-unreachable
> >
> > RETURN ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp time-exceeded
> > match-set i-2-10-VM-6 src
> >
> > ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp time-exceeded
> >
> > RETURN ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp parameter-problem
> > match-set i-2-10-VM-6 src
> >
> > ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> > match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> parameter-problem
> >
> > RETURN ipv6-icmpanywhere ff02::16 PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged
> >
> > RETURN udp  fe80::1c00:f6ff:fe00:56  ff02::1:2PHYSDEV
> > match --physdev-in vnet3 --physdev-is-bridged udp spt:dhcpv6-client
> >
> > ACCEPT udp  fe80::/64fe80::1c00:f6ff:fe00:56  PHYSDEV
> > match --physdev-out vnet3 --physdev-is-bridged udp dpt:dhcpv6-client
> >
> > DROP   udp  anywhere!fe80::/64PHYSDEV
> match
> > --physdev-in 

Re: IPv6 Issue in Cloudstack

[Wido den Hollander]

Can you check with tcpdump on the host and sniff the vnetX device of the
VM to see if you ICMPv6 packages reach the VM?

Security Grouping with IPv6 works with KVM, so it has to be a
configuration issue somewhere.

Wido

On 4/30/21 8:59 PM, Hean Seng wrote:
> Hi
> 
> I am using 4.15 , hypervisor is ubuntu 18 , KVM , yes, I am on advance with
> SG
> 
> I set the Security Group:
> 
> ICMP
> -1 -1 ::/0
> 
> But seems still cannot ping the VM.
> 
> Or even add in rules for ALL
> 
> All .  All   ::/0
> 
> 
> Seems not able to PING.
> 
> 
> After configure , this is the rules in ip6tables
> 
> 
> Chain i-2-10-VM (1 references)
> target prot opt source   destination
> ACCEPT ipv6-icmpanywhere anywhere
> ACCEPT all  anywhere anywhere state NEW
> DROP   all  anywhere anywhere
> 
> 
> 
> 
> Chain i-2-10-VM-eg (1 references)
> 
> target prot opt source   destination
> 
> RETURN all  anywhere anywhere
> 
> 
> Chain i-2-10-def (2 references)
> 
> target prot opt source   destination
> 
> ACCEPT all  anywhere anywhere state
> RELATED,ESTABLISHED
> 
> ACCEPT ipv6-icmpfe80::/64ip6-allnodes PHYSDEV
> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> router-advertisement HL match HL == 255
> 
> RETURN ipv6-icmpanywhere ip6-allrouters   PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp router-solicitation
> HL match HL == 255
> 
> DROP   ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp router-advertisement
> 
> RETURN ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> neighbour-solicitation HL match HL == 255
> 
> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> neighbour-solicitation HL match HL == 255
> 
> RETURN ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> neighbour-advertisement match-set i-2-10-VM-6 src HL match HL == 255
> 
> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> neighbour-advertisement HL match HL == 255
> 
> RETURN ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp packet-too-big
> match-set i-2-10-VM-6 src
> 
> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp packet-too-big
> 
> RETURN ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
> destination-unreachable match-set i-2-10-VM-6 src
> 
> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
> destination-unreachable
> 
> RETURN ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp time-exceeded
> match-set i-2-10-VM-6 src
> 
> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp time-exceeded
> 
> RETURN ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp parameter-problem
> match-set i-2-10-VM-6 src
> 
> ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
> match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp parameter-problem
> 
> RETURN ipv6-icmpanywhere ff02::16 PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged
> 
> RETURN udp  fe80::1c00:f6ff:fe00:56  ff02::1:2PHYSDEV
> match --physdev-in vnet3 --physdev-is-bridged udp spt:dhcpv6-client
> 
> ACCEPT udp  fe80::/64fe80::1c00:f6ff:fe00:56  PHYSDEV
> match --physdev-out vnet3 --physdev-is-bridged udp dpt:dhcpv6-client
> 
> DROP   udp  anywhere!fe80::/64PHYSDEV match
> --physdev-in vnet3 --physdev-is-bridged udp spt:dhcpv6-server
> 
> RETURN udp  anywhere anywhere PHYSDEV match
> --physdev-in vnet3 --physdev-is-bridged udp dpt:domain match-set
> i-2-10-VM-6 src
> 
> RETURN tcp  anywhere anywhere PHYSDEV match
> --physdev-in vnet3 --physdev-is-bridged tcp dpt:domain match-set
> i-2-10-VM-6 src
> 
> DROP   all  anywhere anywhere PHYSDEV match
> --physdev-in vnet3 --physdev-is-bridged ! match-set i-2-10-VM-6 src
> 
> i-2-10-VM-eg  all  anywhere anywhere PHYSDEV
> 

Re: IPv6 Issue in Cloudstack

[Hean Seng]

Hi

I am using 4.15 , hypervisor is ubuntu 18 , KVM , yes, I am on advance with
SG

I set the Security Group:

ICMP
-1 -1 ::/0

But seems still cannot ping the VM.

Or even add in rules for ALL

All .  All   ::/0


Seems not able to PING.


After configure , this is the rules in ip6tables


Chain i-2-10-VM (1 references)
target prot opt source   destination
ACCEPT ipv6-icmpanywhere anywhere
ACCEPT all  anywhere anywhere state NEW
DROP   all  anywhere anywhere




Chain i-2-10-VM-eg (1 references)

target prot opt source   destination

RETURN all  anywhere anywhere


Chain i-2-10-def (2 references)

target prot opt source   destination

ACCEPT all  anywhere anywhere state
RELATED,ESTABLISHED

ACCEPT ipv6-icmpfe80::/64ip6-allnodes PHYSDEV
match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
router-advertisement HL match HL == 255

RETURN ipv6-icmpanywhere ip6-allrouters   PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp router-solicitation
HL match HL == 255

DROP   ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp router-advertisement

RETURN ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
neighbour-solicitation HL match HL == 255

ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
neighbour-solicitation HL match HL == 255

RETURN ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
neighbour-advertisement match-set i-2-10-VM-6 src HL match HL == 255

ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
neighbour-advertisement HL match HL == 255

RETURN ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp packet-too-big
match-set i-2-10-VM-6 src

ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp packet-too-big

RETURN ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp
destination-unreachable match-set i-2-10-VM-6 src

ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp
destination-unreachable

RETURN ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp time-exceeded
match-set i-2-10-VM-6 src

ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp time-exceeded

RETURN ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged ipv6-icmp parameter-problem
match-set i-2-10-VM-6 src

ACCEPT ipv6-icmpanywhere anywhere PHYSDEV
match --physdev-out vnet3 --physdev-is-bridged ipv6-icmp parameter-problem

RETURN ipv6-icmpanywhere ff02::16 PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged

RETURN udp  fe80::1c00:f6ff:fe00:56  ff02::1:2PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged udp spt:dhcpv6-client

ACCEPT udp  fe80::/64fe80::1c00:f6ff:fe00:56  PHYSDEV
match --physdev-out vnet3 --physdev-is-bridged udp dpt:dhcpv6-client

DROP   udp  anywhere!fe80::/64PHYSDEV match
--physdev-in vnet3 --physdev-is-bridged udp spt:dhcpv6-server

RETURN udp  anywhere anywhere PHYSDEV match
--physdev-in vnet3 --physdev-is-bridged udp dpt:domain match-set
i-2-10-VM-6 src

RETURN tcp  anywhere anywhere PHYSDEV match
--physdev-in vnet3 --physdev-is-bridged tcp dpt:domain match-set
i-2-10-VM-6 src

DROP   all  anywhere anywhere PHYSDEV match
--physdev-in vnet3 --physdev-is-bridged ! match-set i-2-10-VM-6 src

i-2-10-VM-eg  all  anywhere anywhere PHYSDEV
match --physdev-in vnet3 --physdev-is-bridged match-set i-2-10-VM-6 src

i-2-10-VM  all  anywhere anywhere PHYSDEV match
--physdev-out vnet3 --physdev-is-bridged





On Sat, May 1, 2021 at 1:42 AM Gabriel Bräscher 
wrote:

> Hi Hean,
>
> What version of CloudStack are you using?
>
> KVM does support IPv6 indeed when deploying Advanced Networking with
> Security Groups (SG) enabled.
> It should work fine. The only difference regarding setting IPv4 rules for
> SG is that the CIDR 

Re: IPv6 Issue in Cloudstack

[Hean Seng]

We using share network,  on Security Group, KVM .

On Fri, Apr 30, 2021 at 6:28 PM Alex Mattioli 
wrote:

> Hi Hean,
>
> What type of network and hypervisor are you using? Also, which version of
> ACS?
>
> Regards,
> Alex
>
>
>
>
>
> -Original Message-
> From: Hean Seng 
> Sent: 30 April 2021 08:34
> To: users@cloudstack.apache.org
> Subject: IPv6 Issue in Cloudstack
>
> Hi
>
> I setup the IPv6 in VM.  Outbound form VM is no issue, can ping all the
> Ipv6 ip outside .
>
> But Inboud th IPv6 IP in VM seems all not accessible .
>
> And seem there no Security Group to manange the IPv6 rules . The SG is
> only for IPv4.
>
> and I saw ipv6tables -L , there is a lot of rules there .  Not sure is
> preconfigured by Cloudstack or Default Linux. And I guess that is blocking
> access
>
> Anybody have experience on enabling IPv6 in Cloudstack VM and the
> Ipv6table rules there ?
>
>
> --
> Regards,
> Hean Seng
>


-- 
Regards,
Hean Seng

RE: IPv6 Issue in Cloudstack

[Alex Mattioli]

Hi Hean,

What type of network and hypervisor are you using? Also, which version of ACS?

Regards,
Alex


 


-Original Message-
From: Hean Seng  
Sent: 30 April 2021 08:34
To: users@cloudstack.apache.org
Subject: IPv6 Issue in Cloudstack

Hi

I setup the IPv6 in VM.  Outbound form VM is no issue, can ping all the
Ipv6 ip outside .

But Inboud th IPv6 IP in VM seems all not accessible .

And seem there no Security Group to manange the IPv6 rules . The SG is only for 
IPv4.

and I saw ipv6tables -L , there is a lot of rules there .  Not sure is 
preconfigured by Cloudstack or Default Linux. And I guess that is blocking 
access

Anybody have experience on enabling IPv6 in Cloudstack VM and the Ipv6table 
rules there ?


--
Regards,
Hean Seng

Re: IPv6 Support

[Hean Seng]

Hi Gabriel.

For this case, :

n such a network setup it is possible to deploy multiple shared guest
networks, isolated via VLAN/VXLAN. These networks can be configured with
only IPv4 addresses, or IPv4 + IPv6; on the second case the IPv4 address
could be a either a private IP (e.g. 10.1.1.1) or a public IP; all VMs then
have a public IPv6 address.

For those have IPv4+IPv6 , can I know how you configure it ?


On Wed, Nov 11, 2020 at 10:26 PM Hean Seng  wrote:

> For ipv6 implementation for Advancezone with NAT,  i guess shall be
> allocate a ipv6 /64 subnet to it  (the Virtual Router), and  VirtualRouter
> allocate IPv6 to  VM under it.
> So cloudstack shall allow add ipv6 /64 subnet to the zone , and when VM
> created , it will assign a /64 subnet to VR, and VR have DHCP6 to
> allocate IP to the VM.
>
> On Wed, Nov 11, 2020 at 7:54 PM Eric Lee Green 
> wrote:
>
>> On 11/11/2020 2:01 AM, Hean Seng wrote:
>> > IPv6 do not have NAT , each VM suppose to have indiviual Ipv6 Address.
>>
>> NAT66 does in fact exist, and the virtual routers used for VLANs could
>> in fact be configured with RADV to provide an IETF RFC4193 SLAAC prefix
>> to private VPC networks then use NAT66 to communicate with the rest of
>> the IPv6 Internet via a SLAAC-configured IPv6 address on the virtual
>> router's public interface. They are not currently so configured, but all
>> the stuff to do it is already there in the base Debian distribution used
>> for the virtual routers.
>>
>> Port forwarding would require changes to the virtual router to allow
>> IPv6 port forwarding (as well as likely allowing a fixed IPv6 address
>> for the virtual router rather than SLAAC).
>>
>> DHCPv6 to advertise IPv6 DNS servers would be the other part of that
>> equation.
>>
>> Routing public subnets would require significant work, since the virtual
>> routers would need to advertise routes upstream to whatever layer 3
>> switch or router routes things to and from the Internet. In addition
>> security would require disabling incoming IPv6 connections to the
>> advertised subnet except to specific instances that have a hole poked in
>> the firewall allowing incoming IPv6. It is unlikely that anybody is
>> going to bother implementing this anytime soon, since NAT66 works fine
>> for Cloudstack's purposes and is significantly easier to implement since
>> it doesn't require upstream routers to accept route advertisements from
>> virtual routers.
>>
>> >
>> > For NAT zone,  is that any way to allocate IPv6 subnet ?
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Tue, Nov 10, 2020 at 3:51 PM Andrija Panic 
>> > wrote:
>> >
>> >> If not mistaken, ipv6 is only supported for Shared Networks, and not
>> for
>> >> Isolated/VPC networks.
>> >>
>> >> On Tue, 3 Nov 2020 at 04:31, Hean Seng  wrote:
>> >>
>> >>> Hi
>> >>>
>> >>> Is that anyone have a idea of best way implementing ipv6 in
>> cloudstack ?
>> >>>
>> >>> I saw the doc, and mentioned create another SharedGuestNework in
>> >>> AdvanceZone, and assigned ipv6 /64 network there.
>> >>>
>> >>> However, I not quite understand is in Advancezone with NAT (public ip,
>> >>> isolated vlan), the network of  the VM is  their own LAN IP and
>> isolated
>> >> by
>> >>> VLAN or VXLAN.   How can we assign Ipv6 over there? Or shall we
>> >> create
>> >>> another SharedGuestNetwork with another VLAN , and assign another
>> >>> GuestNetwork manually to the VM ?  But then, the VM become 2
>> network.  Is
>> >>> that the way to do ?
>> >>>
>> >>>
>> >>> --
>> >>> Regards,
>> >>> Hean Seng
>> >>>
>> >>
>> >> --
>> >>
>> >> Andrija Panić
>> >>
>> >
>>
>
>
> --
> Regards,
> Hean Seng
>


-- 
Regards,
Hean Seng

Re: IPv6 Support

[Hean Seng]

For ipv6 implementation for Advancezone with NAT,  i guess shall be
allocate a ipv6 /64 subnet to it  (the Virtual Router), and  VirtualRouter
allocate IPv6 to  VM under it.
So cloudstack shall allow add ipv6 /64 subnet to the zone , and when VM
created , it will assign a /64 subnet to VR, and VR have DHCP6 to
allocate IP to the VM.

On Wed, Nov 11, 2020 at 7:54 PM Eric Lee Green 
wrote:

> On 11/11/2020 2:01 AM, Hean Seng wrote:
> > IPv6 do not have NAT , each VM suppose to have indiviual Ipv6 Address.
>
> NAT66 does in fact exist, and the virtual routers used for VLANs could
> in fact be configured with RADV to provide an IETF RFC4193 SLAAC prefix
> to private VPC networks then use NAT66 to communicate with the rest of
> the IPv6 Internet via a SLAAC-configured IPv6 address on the virtual
> router's public interface. They are not currently so configured, but all
> the stuff to do it is already there in the base Debian distribution used
> for the virtual routers.
>
> Port forwarding would require changes to the virtual router to allow
> IPv6 port forwarding (as well as likely allowing a fixed IPv6 address
> for the virtual router rather than SLAAC).
>
> DHCPv6 to advertise IPv6 DNS servers would be the other part of that
> equation.
>
> Routing public subnets would require significant work, since the virtual
> routers would need to advertise routes upstream to whatever layer 3
> switch or router routes things to and from the Internet. In addition
> security would require disabling incoming IPv6 connections to the
> advertised subnet except to specific instances that have a hole poked in
> the firewall allowing incoming IPv6. It is unlikely that anybody is
> going to bother implementing this anytime soon, since NAT66 works fine
> for Cloudstack's purposes and is significantly easier to implement since
> it doesn't require upstream routers to accept route advertisements from
> virtual routers.
>
> >
> > For NAT zone,  is that any way to allocate IPv6 subnet ?
> >
> >
> >
> >
> >
> >
> >
> > On Tue, Nov 10, 2020 at 3:51 PM Andrija Panic 
> > wrote:
> >
> >> If not mistaken, ipv6 is only supported for Shared Networks, and not for
> >> Isolated/VPC networks.
> >>
> >> On Tue, 3 Nov 2020 at 04:31, Hean Seng  wrote:
> >>
> >>> Hi
> >>>
> >>> Is that anyone have a idea of best way implementing ipv6 in cloudstack
> ?
> >>>
> >>> I saw the doc, and mentioned create another SharedGuestNework in
> >>> AdvanceZone, and assigned ipv6 /64 network there.
> >>>
> >>> However, I not quite understand is in Advancezone with NAT (public ip,
> >>> isolated vlan), the network of  the VM is  their own LAN IP and
> isolated
> >> by
> >>> VLAN or VXLAN.   How can we assign Ipv6 over there? Or shall we
> >> create
> >>> another SharedGuestNetwork with another VLAN , and assign another
> >>> GuestNetwork manually to the VM ?  But then, the VM become 2 network.
> Is
> >>> that the way to do ?
> >>>
> >>>
> >>> --
> >>> Regards,
> >>> Hean Seng
> >>>
> >>
> >> --
> >>
> >> Andrija Panić
> >>
> >
>


-- 
Regards,
Hean Seng

Re: IPv6 Support

[Gabriel Beims Bräscher]

I might get a bit redundant here, but here follows my 2 cents on IPv6 +
CloudStack.

Andrija is right. IPv6 works on Zones with Advanced Network + Security
Groups + KVM.
The documentation [1] also raised support provided by XenServer, but to be
honest I have no experience with IPv6 + XenServer to comment about it.

To deploy IPv6 networks, you must deploy a Zone with advanced network +
security group setting the IPV6 fileds (DNS); if IPv6 enabled networks are
created but the Zone hasn't IPv6 DNS1 or DNS2 configured then *dnsmasq*
inside the Virtual Router does not start.

In such a network setup it is possible to deploy multiple shared guest
networks, isolated via VLAN/VXLAN. These networks can be configured with
only IPv4 addresses, or IPv4 + IPv6; on the second case the IPv4 address
could be a either a private IP (e.g. 10.1.1.1) or a public IP; all VMs then
have a public IPv6 address.

CloudStack IPv6 + Security Group is implemented using Stateless address
autoconfiguration (*SLAAC*), which requires each network to have a /64
address block; nat/port forwarding is not necessary therefore.

Why using Security Group? This happens due to the fact that so far all IPv6
ACLs are handled by implementations on hypervisor (security group
implementation) instead of VRs/VPCs.

Eric Lee Green is right as well; I don't see anyone implementing IPv6 for
NAT. Implementing it on VR is possible but adds quite a lot of complexity,
it would be easier to have a mix of both worlds; e.g. NAT, VPCs for IPv4
networks, and Security Groups for IPv6 networks using SLAAC.

[1] http://docs.cloudstack.apache.org/en/latest/plugins/ipv6.html

Em qua., 11 de nov. de 2020 às 08:54, Eric Lee Green <
eric.lee.gr...@gmail.com> escreveu:

> On 11/11/2020 2:01 AM, Hean Seng wrote:
> > IPv6 do not have NAT , each VM suppose to have indiviual Ipv6 Address.
>
> NAT66 does in fact exist, and the virtual routers used for VLANs could
> in fact be configured with RADV to provide an IETF RFC4193 SLAAC prefix
> to private VPC networks then use NAT66 to communicate with the rest of
> the IPv6 Internet via a SLAAC-configured IPv6 address on the virtual
> router's public interface. They are not currently so configured, but all
> the stuff to do it is already there in the base Debian distribution used
> for the virtual routers.
>
> Port forwarding would require changes to the virtual router to allow
> IPv6 port forwarding (as well as likely allowing a fixed IPv6 address
> for the virtual router rather than SLAAC).
>
> DHCPv6 to advertise IPv6 DNS servers would be the other part of that
> equation.
>
> Routing public subnets would require significant work, since the virtual
> routers would need to advertise routes upstream to whatever layer 3
> switch or router routes things to and from the Internet. In addition
> security would require disabling incoming IPv6 connections to the
> advertised subnet except to specific instances that have a hole poked in
> the firewall allowing incoming IPv6. It is unlikely that anybody is
> going to bother implementing this anytime soon, since NAT66 works fine
> for Cloudstack's purposes and is significantly easier to implement since
> it doesn't require upstream routers to accept route advertisements from
> virtual routers.
>
> >
> > For NAT zone,  is that any way to allocate IPv6 subnet ?
> >
> >
> >
> >
> >
> >
> >
> > On Tue, Nov 10, 2020 at 3:51 PM Andrija Panic 
> > wrote:
> >
> >> If not mistaken, ipv6 is only supported for Shared Networks, and not for
> >> Isolated/VPC networks.
> >>
> >> On Tue, 3 Nov 2020 at 04:31, Hean Seng  wrote:
> >>
> >>> Hi
> >>>
> >>> Is that anyone have a idea of best way implementing ipv6 in cloudstack
> ?
> >>>
> >>> I saw the doc, and mentioned create another SharedGuestNework in
> >>> AdvanceZone, and assigned ipv6 /64 network there.
> >>>
> >>> However, I not quite understand is in Advancezone with NAT (public ip,
> >>> isolated vlan), the network of  the VM is  their own LAN IP and
> isolated
> >> by
> >>> VLAN or VXLAN.   How can we assign Ipv6 over there? Or shall we
> >> create
> >>> another SharedGuestNetwork with another VLAN , and assign another
> >>> GuestNetwork manually to the VM ?  But then, the VM become 2 network.
> Is
> >>> that the way to do ?
> >>>
> >>>
> >>> --
> >>> Regards,
> >>> Hean Seng
> >>>
> >>
> >> --
> >>
> >> Andrija Panić
> >>
> >
>

Re: IPv6 Support

[Eric Lee Green]

On 11/11/2020 2:01 AM, Hean Seng wrote:

IPv6 do not have NAT , each VM suppose to have indiviual Ipv6 Address.


NAT66 does in fact exist, and the virtual routers used for VLANs could 
in fact be configured with RADV to provide an IETF RFC4193 SLAAC prefix 
to private VPC networks then use NAT66 to communicate with the rest of 
the IPv6 Internet via a SLAAC-configured IPv6 address on the virtual 
router's public interface. They are not currently so configured, but all 
the stuff to do it is already there in the base Debian distribution used 
for the virtual routers.


Port forwarding would require changes to the virtual router to allow 
IPv6 port forwarding (as well as likely allowing a fixed IPv6 address 
for the virtual router rather than SLAAC).


DHCPv6 to advertise IPv6 DNS servers would be the other part of that 
equation.


Routing public subnets would require significant work, since the virtual 
routers would need to advertise routes upstream to whatever layer 3 
switch or router routes things to and from the Internet. In addition 
security would require disabling incoming IPv6 connections to the 
advertised subnet except to specific instances that have a hole poked in 
the firewall allowing incoming IPv6. It is unlikely that anybody is 
going to bother implementing this anytime soon, since NAT66 works fine 
for Cloudstack's purposes and is significantly easier to implement since 
it doesn't require upstream routers to accept route advertisements from 
virtual routers.




For NAT zone,  is that any way to allocate IPv6 subnet ?







On Tue, Nov 10, 2020 at 3:51 PM Andrija Panic 
wrote:


If not mistaken, ipv6 is only supported for Shared Networks, and not for
Isolated/VPC networks.

On Tue, 3 Nov 2020 at 04:31, Hean Seng  wrote:


Hi

Is that anyone have a idea of best way implementing ipv6 in cloudstack ?

I saw the doc, and mentioned create another SharedGuestNework in
AdvanceZone, and assigned ipv6 /64 network there.

However, I not quite understand is in Advancezone with NAT (public ip,
isolated vlan), the network of  the VM is  their own LAN IP and isolated

by

VLAN or VXLAN.   How can we assign Ipv6 over there? Or shall we

create

another SharedGuestNetwork with another VLAN , and assign another
GuestNetwork manually to the VM ?  But then, the VM become 2 network.  Is
that the way to do ?


--
Regards,
Hean Seng



--

Andrija Panić

Re: IPv6 Support

[Hean Seng]

IPv6 do not have NAT , each VM suppose to have indiviual Ipv6 Address.

For NAT zone,  is that any way to allocate IPv6 subnet ?







On Tue, Nov 10, 2020 at 3:51 PM Andrija Panic 
wrote:

> If not mistaken, ipv6 is only supported for Shared Networks, and not for
> Isolated/VPC networks.
>
> On Tue, 3 Nov 2020 at 04:31, Hean Seng  wrote:
>
> > Hi
> >
> > Is that anyone have a idea of best way implementing ipv6 in cloudstack ?
> >
> > I saw the doc, and mentioned create another SharedGuestNework in
> > AdvanceZone, and assigned ipv6 /64 network there.
> >
> > However, I not quite understand is in Advancezone with NAT (public ip,
> > isolated vlan), the network of  the VM is  their own LAN IP and isolated
> by
> > VLAN or VXLAN.   How can we assign Ipv6 over there? Or shall we
> create
> > another SharedGuestNetwork with another VLAN , and assign another
> > GuestNetwork manually to the VM ?  But then, the VM become 2 network.  Is
> > that the way to do ?
> >
> >
> > --
> > Regards,
> > Hean Seng
> >
>
>
> --
>
> Andrija Panić
>


-- 
Regards,
Hean Seng

Re: IPv6 Support

[Andrija Panic]

If not mistaken, ipv6 is only supported for Shared Networks, and not for
Isolated/VPC networks.

On Tue, 3 Nov 2020 at 04:31, Hean Seng  wrote:

> Hi
>
> Is that anyone have a idea of best way implementing ipv6 in cloudstack ?
>
> I saw the doc, and mentioned create another SharedGuestNework in
> AdvanceZone, and assigned ipv6 /64 network there.
>
> However, I not quite understand is in Advancezone with NAT (public ip,
> isolated vlan), the network of  the VM is  their own LAN IP and isolated by
> VLAN or VXLAN.   How can we assign Ipv6 over there? Or shall we create
> another SharedGuestNetwork with another VLAN , and assign another
> GuestNetwork manually to the VM ?  But then, the VM become 2 network.  Is
> that the way to do ?
>
>
> --
> Regards,
> Hean Seng
>


-- 

Andrija Panić

Re: IPv6

[Simon Weller]

Have you checked whether there is a Jira issue raised for it?



From: uabstarn...@gmail.com  on behalf of Mindaugas 
Milinavicius 
Sent: Tuesday, September 20, 2016 7:53 AM
To: users@cloudstack.apache.org
Subject: IPv6

Hello,

anyone know then issue with IPv6 CIDR will be fixed? It taking too long...



Pagarbiai
Mindaugas Milinavicius
UAB STARNITA
Direktorius
http://www.clustspace.com
LT: +37068882880
RU: +7913933

Tomorrow's possibilities today


   - 1 core CPU, 512MB RAM, 20GB (EUR 5.00)
   - 1 core CPU, 1GB RAM, 30GB (EUR 10.00)
   - 2 core CPU, 2GB RAM, 40GB (EUR 20.00)
   - 2 core CPU, 4GB RAM, 60GB (EUR 40.00)
   - 4 core CPU, 8GB RAM, 80GB (EUR 80.00)

RE: IPv6

[Marty Godsey]

Let me know how it goes. I have a /48 assigned to me so I can easily divide it 
into /64s I just have been running into issues actually putting in ACS.

This what I posted in the Slack channel:

"I am implementing IPv6 for my clients and want to add it as a shared network 
for allocation. However, Cloudstack doesn't seem happy with it. I have tried 
adding it via the GUI (/64s, which according to the documentation is the 
"smallest" supported) as well as via cloudmonkey and it tells me I have to have 
an advance networking zone, which I do.."

Regards,
Marty Godsey

-Original Message-
From: uabstarn...@gmail.com [mailto:uabstarn...@gmail.com] On Behalf Of 
Mindaugas Milinavicius
Sent: Thursday, September 1, 2016 1:47 PM
To: users@cloudstack.apache.org
Subject: Re: IPv6

i will integrate ipv6 in next week




Pagarbiai
Mindaugas Milinavičius
UAB STARNITA
Direktorius
http://www.clustspace.com
LT: +37068882880
RU: +7913933

Tomorrow's possibilities today
<http://www.clustspace.com/>

   - 1 core CPU, 512MB RAM, 20GB (€ 5.00)
   - 1 core CPU, 1GB RAM, 30GB (€ 10.00)
   - 2 core CPU, 2GB RAM, 40GB (€ 20.00)
   - 2 core CPU, 4GB RAM, 60GB (€ 40.00)
   - 4 core CPU, 8GB RAM, 80GB (€ 80.00)


On Thu, Sep 1, 2016 at 8:42 PM, Simon Weller <swel...@ena.com> wrote:

> Marty,
>
>
> I know some folks in Europe are using v6, but I'm not sure to what degree.
> We haven't started looking at it yet, but we will at some point.
>
>
> - Si
>
> 
> From: Marty Godsey <ma...@gonsource.com>
> Sent: Wednesday, August 31, 2016 2:48 PM
> To: users@cloudstack.apache.org
> Subject: IPv6
>
> So.. The subject says it all.. :) to a degree.
>
> Has anyone used/using IPv6 in Cloudstack and based on the support 
> docs, It has to be a /64? I can subnet to say a /109?
>
> Regards,
> Marty Godsey
>
>

Re: IPv6

[Mindaugas Milinavičius]

i will integrate ipv6 in next week




Pagarbiai
Mindaugas Milinavičius
UAB STARNITA
Direktorius
http://www.clustspace.com
LT: +37068882880
RU: +7913933

Tomorrow's possibilities today


   - 1 core CPU, 512MB RAM, 20GB (€ 5.00)
   - 1 core CPU, 1GB RAM, 30GB (€ 10.00)
   - 2 core CPU, 2GB RAM, 40GB (€ 20.00)
   - 2 core CPU, 4GB RAM, 60GB (€ 40.00)
   - 4 core CPU, 8GB RAM, 80GB (€ 80.00)


On Thu, Sep 1, 2016 at 8:42 PM, Simon Weller  wrote:

> Marty,
>
>
> I know some folks in Europe are using v6, but I'm not sure to what degree.
> We haven't started looking at it yet, but we will at some point.
>
>
> - Si
>
> 
> From: Marty Godsey 
> Sent: Wednesday, August 31, 2016 2:48 PM
> To: users@cloudstack.apache.org
> Subject: IPv6
>
> So.. The subject says it all.. :) to a degree.
>
> Has anyone used/using IPv6 in Cloudstack and based on the support docs, It
> has to be a /64? I can subnet to say a /109?
>
> Regards,
> Marty Godsey
>
>

Re: IPv6

[Simon Weller]

Marty,


I know some folks in Europe are using v6, but I'm not sure to what degree. We 
haven't started looking at it yet, but we will at some point.


- Si


From: Marty Godsey 
Sent: Wednesday, August 31, 2016 2:48 PM
To: users@cloudstack.apache.org
Subject: IPv6

So.. The subject says it all.. :) to a degree.

Has anyone used/using IPv6 in Cloudstack and based on the support docs, It has 
to be a /64? I can subnet to say a /109?

Regards,
Marty Godsey

Re: IPv6 support for ESXi?

[sebgoa]

Hugo,

Do you know the status of IPv6 ?

On Oct 8, 2014, at 10:48 AM, Octavian Popescu octavian.pope...@interoute.com 
wrote:

 Hi,
 
 Is there any IPv6 support in CS for ESXi hypervisors? I've read the 
 documentation but only KVM and Xen are mentioned so I was wondering if this 
 will change in the new releases.
 
 Thanks,
 Octavian
 
 
 

Re: IPv6 Prefix

[Erik Weber]

On Mon, Aug 4, 2014 at 12:16 PM, Soeren Malchow soeren.malc...@mcon.net
wrote:

 Dear all,

 is it possible that Cloudstack 4.4 can only use /64 prefixes for Ipv6 ?

 We are trying to use /96 prefixes and it complains exactly about that.

 If that is only a frontend thing, can we type in /64 and change it in the
 database later on ?



I don't think it's a frontend only issue.

http://docs.cloudstack.apache.org/en/latest/networking/ipv6.html

-- 
Erik