Daniel, Dag very well explained the problem with VMware and Virtual Guest Tagging (VGT). I could add to that, if you'd use a Distributed Virtual Switch (DVS) you effectively can limit tags on a trunked connection to a Vmware Guest.
So you would need to: (1) Use VMware DVS (Enterprise Plus Feature Set) (2) Adapt CloudStack Source to use VGT to vRouter in DVS Mode (3) Update your vmware-template or provisioning of vRouter to use .1q On Tue, Aug 15, 2017 at 3:34 PM, Dag Sonstebo <dag.sonst...@shapeblue.com> wrote: > Hi Daniel, > > Yes you could do .1q at an interface level for the VR ( this is what we do > with KVM networking ). However this brings you a couple of stumbling blocks: > > 1) For you to trunk VLANs to this interface it would need to be attached > to a trunked vSwitch – which is currently all or nothing in VMware (by > setting vlan for the vSwitch to 4095) – i.e. you can’t set a vSwitch to > only trunk certain VLAN ranges. This now brings you a further problem – if > you did trunk at the vSwitch level you would have to configure your top of > rack switches to do the same. Again – this is possible – but when you > consider that you *must* be able to isolate all VLAN traffic on a per > CloudStack account level – this would mean you would need one or more ESXi > physical NICs per account + a considerable count of top of rack physical > switch ports. So – you are effectively moving the problem from the virtual > switches to the physical ones, which while technically possible is not > feasible. > > 2) Your main problem is at the user VM end. If we agree you can’t expect > VLAN tags to be set at the guest OS level then the only other place to set > this is at the vSwitch level. Keeping in mind the limitations mentioned > above this means you need at least one vSwich ( = VLAN ) per VPC tier. > Since there is no way other than the all-in trunking mentioned above for > the VPC VR to connect to all of these tier vSwitches implementing .1q at > the VR level would not work. > > Keep in mind though – my points above are purely in the context of VMware > and VLANs – as soon as you step into the SDN world you move to overlay > networks etc where other mechanisms could be and are implemented. > > = = > > Dennis – to answer your question as well – CloudStack speaks to XenServer > using the API. I think you have probably answered your own question though > – as you pointed out in your discussion forum thread all documentation says > 7 NICs is the max supported. If you do some testing and find the API can > handle more than 7 then I would suggest to log a Jira ticket such that this > can be implemented in future CloudStack versions. > > > Regards, > Dag Sonstebo > Cloud Architect > ShapeBlue > > On 15/08/2017, 14:10, "daniel.herrm...@zv.fraunhofer.de" < > daniel.herrm...@zv.fraunhofer.de> wrote: > > Hi Dag, > > you would need to do that with the Linux dot1q kernel module, yes. > This way you can create virtual interfaces with VLAN tags and bind them to > one NIC. We are routing and firewalling in software anyway, I do not see > any considerable additional overhead here. Instead of “physical” NICs, we > have one of them and create the other as VLAN interface. > > I do not really understand the security problems as well. No user is > ever expected to have access to the virtual router. So how would that > affect security? > > Regards > Daniel > > Am 15.08.17, 14:36 schrieb "Dag Sonstebo" <dag.sonst...@shapeblue.com > >: > > Hi Daniel, > > The mechanism for isolating L2 traffic is at the vSwitch level – > there is no way to VLAN tag the at the NIC level for a VM in VMware. Your > only other option is therefore to VLAN tag at the guest OS level which adds > security issues + overhead, etc. > > Regards, > Dag Sonstebo > Cloud Architect > ShapeBlue > > On 15/08/2017, 13:05, "daniel.herrm...@zv.fraunhofer.de" < > daniel.herrm...@zv.fraunhofer.de> wrote: > > Hi Dag, > > thank you for your answer. As far as I know, the end user > never has direct access to the virtual router. I am not talking about > adding a VLAN tag at the user VM, only at the VPR, where the limit most > likely comes into play when creating a number of tiers in a VPC. > > We could do both: normal VMs require one interface per > tier/network, which makes perfect sense. The router however could use VLAN > tags at VM level, which could remove the limitation of having a maximum > number of tiers connected to one VPC. It is only configured by CloudStack, > the end user does not have access to the VPR. > > Regards > Daniel > > Am 15.08.17, 13:27 schrieb "Dag Sonstebo" < > dag.sonst...@shapeblue.com>: > > Hi Daniel, > > In theory that could work – but keep in mind we are > working in a multi-tenant environment, where guest isolation must be > guaranteed, hence cannot ever be exposed to normal users. The isolation > method must be abstracted from the end user VMs – otherwise you would have > a potential security issue where someone could tag traffic from their VM > with someone else’s tag. Doing tagging at VM level would also be a huge > overhead. > As a result we VLAN tag at the vSwitch or bridge level – > which end users have no access to – the flipside of the coin being that > this requires separate NICs for each tier. > > Regards, > Dag Sonstebo > Cloud Architect > ShapeBlue > > On 15/08/2017, 11:07, "daniel.herrm...@zv.fraunhofer.de" < > daniel.herrm...@zv.fraunhofer.de> wrote: > > Hi, > > we are hitting the same limitation, except that we can > use 10 NICs on VMware. > > The fact that we also use the Private Gateway > functionality addes another NIC, besides the management and outside NIC > which is present as well. > > I wonder that is the reason for one NIC per tier? Why > not just use one outside NIC, one management NIC and *one* NIC for the > tiers, where the VLANs (or whatever isolation method is used) is trunked, > for example just using subinterfaces and dot1Q tags? This would eliminate > this limit for whatever hypervisor that supports trunk to it’s guests (I > know for sure about VMWare, not so much about the other hypervisors). > > Regards > Daniel > > Am 15.08.17, 10:52 schrieb "Dag Sonstebo" < > dag.sonst...@shapeblue.com>: > > Hi Dennis, > > Any tier or network which is accessible and part > of a VPC requires an interface on the VPC Virtual Router. > > What you can however do is create separate shared > networks and connect these as secondary networks to your VMs – these shared > networks get their own VR. > > Regards, > Dag Sonstebo > Cloud Architect > ShapeBlue > > On 15/08/2017, 09:19, "Dennis Meyer" < > snooop...@gmail.com> wrote: > > Hi, > > im using xenserver as hypervisor so im limited > to 7 nic's / vm, so the > router vm cant handle more than 7 nics which > corresponds to 7 networks > inside a vpc. I had created some networks for > different drbd and corosync > stuff, they dont need a gateway, dhcp and a > router vm. How should a network > offering look like which dont creates a > network on the routervm but is > accessible by the vpc? > > Snooops > > > > dag.sonst...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > > > > > > > dag.sonst...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > > > > > > > dag.sonst...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > > > > > > > dag.sonst...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > >
