I am going to try to do a bit of a brain dump from me and my team in this post, and hopefully I can verbalize the ideas well enough that you can all follow along. Please provide feedback, ideas and suggestions...
*Preface:* *--------* I am currently approaching this problem with the following use cases in mind. - Stabilize/replace the VR for our production cloud deployments as well as give the community a free, open source and production grade alternative to the current VR. - I have been scoping the integration of the Fortinet Fortigate (hardware and virtual) into ACS to support the following features. -- All Isolated Guest Network traffic features (Firewall, PF, NAT, Client-to-Site VPN, etc) -- All VPC network traffic features (ACLs, PF, NAT, Client-to-Site and Site-to-Site VPN, Private Gateway) -- Public IPv6 support (NAT64 and NAT46) with IPv4 on the guest side. *The Setup:* *----------* I am not going to get into the details of the Fortinet integration details in this thread, you can expect a function spec from me soon describing my plans for this work. I am currently waiting on hardware to validate my implementation strategy. In this post I will be focusing on the way ACS deals with the VR, network plugins and how this could be potentially adapted to give us more flexibility going forward. I have created a diagram [1] to outline the ideas presented in this post. So if you don't like my ideas here, you can simply use the VR as you currently do and you can basically ignore the dotted circles to the right of the 'existing VR' in the solid circle. The basic idea here, is when you create a network service offering and you select different Providers to handle different network capabilities, you can also specify a VM template for those providers (as well as service offering) which will be used for additional VMs which will be created to provision those services. In this context, think of templates such as VyOS, FortiOS, or a VPX to understand the spirit of this approach. When a network is provisioned, the current VR will be deployed (ideally only offering services like; password server, userdata, DHCP, DNS), and in addition, any network provider templates which have been specified will also be provisioned. Each 'network VM' will get a leg on both the public VLAN and the guest VLAN. In the case of a VPC, the first tier will be a little special (I will get into this in a bit) and will not be able to be deleted until the VR(s) are all cleaned up and the network is completely destroyed. This network is denoted as the 'Client MGMT' network in the diagram [1]. You will notice that ONLY the 'existing VR' has network connectivity to the ACS MGMT server. It may not necessarily be through a 169.x.x.x address, but that illustrates the point for now. The reason for this is because we can not guarantee that a Guest User does may not gain access to these VRs, and in cases I will get into later, the Guest User may actually be given access to those VRs. This does however mean that the ACS MGMT server does not have direct access to those VRs in order to configure them. To solve this, the 'existing VR' would act as a proxy for the ACS Management server over the ACS MGMT network to pass configuration commands to the VRs to the right of the 'existing VR' depicted in the graphic. *The Details:* *------------* Now that you have a basic idea of the deployment structure I have in mind, let's discuss some of the finer details. - We should be able to relatively easily adapt existing external network plugins to be able to be provisioned with this mechanism (assuming they have a VM option). The Palo Alto would be an example which I would likely use as a proof of concept for this adaptation. It would be important that the external plugin could work as an external physical appliance OR as a dynamically provisioned VM appliance. - Enterprise grade network appliances which support VM deployment will no longer have to be pre-deployed and shared, but instead will be able to be dynamically provisioned on a network by network basis. - Existing external network appliance plugins targeting physical boxes will still work in concert with this approach without any need for modification. - We could add an 'Allow Guest Access' checkbox to the service provider section in the network service offering to give the Guest User access to their network appliance. This is especially useful for an implementation like the FortiGate because it allows for the user to configure UTM features which ACS does not orchestrate but the appliance is capable of. This also applies to devices like the NetScaler. In the diagram [1] this is depicted by the dashed green lines between the devices and the 'Client MGMT' network. So in the case of the NetScaler, you would get an NSIP on the Client MGMT network which would allow the Guest User to connect to that box. I envision a URL pointing to the guest network IP with username/password returned for a network which a user can use to connect into that network appliance. - Similar to above, the Guest User will have much more visibility into the logging and access on the network. I can't count how many times a customer has asked for an easy way to get the IPsec logs from the VR. I think this is enough to get the idea across. I have worked through a bunch of examples on a white board with my team to try to work through a bunch of the use cases, edge cases and deployment scenarios, but I am sure there are situations we have not thought through yet. The current VR has been a constant pain for us and our service customers, so it is likely that we will be taking some action one way or another because this needs to be addressed. I know there is a lot going on here, but hopefully I have been able to get our ideas across. Please ask questions or make suggestions. Are there major pieces we are not taking into account? Do you have use cases which would be simplified by this approach that I have not called out? Would love to hear your feedback... [1] https://objects-east.cloud.ca/v1/5ef827605f884961b94881e928e7a2 50/swill/vr_network.png *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Sep 20, 2016 at 5:54 AM, Murali Reddy <muralimmre...@gmail.com> wrote: > Configuration management of network appliances particularly for Cloud and > NFV scenarios is still evolving area. Programmability is the not the > strength for even the most popular network operating systems like IOS, > JunoS etc. So its not surprising why CloudStack integrates in a archaic way > with stock linux for the VR. > > VR was never integral/hardcoded option in CloudStack. Its always been a > plugin. CloudStack network orchestration is well abstracted and designed > with vision to compose a network with different set of providers for > different services. Yes that vision is not fully realised yet, and we don’t > have true service function chains. That would be different discussion topic. > > I tend to agree with Simon, as alternate/interim option we can take hard > look whats causing the problems with current VR integration. Personally, I > think it would be easier we take a cue from configuration managers and > network configuration solutions out there (for e.g promise theory based > Cisco ACI) move to more declarative model of expressing desired state of > network configuration. Infact current VR from 4.6, actually holds the > desired state per service basis, seems to be in that direction. > > It does make sense to evaluate new appliances which can provide rich > semantics (like programmable API, declarative configuration, versioning, > commit/rollback etc), but will need significant engineering effort and time > to stabilise. We may make same mistakes with integration of other appliance > as well, if we fully don’t understand the nature of the current problems > with CloudStack core and service provider interaction and current VR > integration. > > > On 16/09/16, 11:59 PM, "Simon Weller" <swel...@ena.com> wrote: > > >I think our other option is to take a real look at what it would take to > fix the VR. In my opinion, a lot of the problems are related to the > monolithic python code base and the fact nothing is actually separated. > > > >Secondly, the python scripts (and bash scripts) don't use any established > libraries to complete tasks and instead shell out and run commands that are > both hard to track and hard to parse on return. > > > > > >If we daemonized this, used a real api for Agent to VR communication, > used common already existing libraries for the system service and network > interactions and spent a bit of time separating out code into distinct > modules, everything would behave a lot better. > > > > > >The pain and suffering is due to years and years of patches and constant > shelling out to complete tasks in my opinion. If we spend time to rethink > how we interact with the VR in general and we abstract the systems and > networking stuff and use well known and stable libraries to do the work, > the VR would be much easier to maintain. > > > > > >- Si > > > > > > > > > >________________________________ > >From: Marty Godsey <ma...@gonsource.com> > >Sent: Friday, September 16, 2016 12:24 PM > >To: dev@cloudstack.apache.org > >Subject: RE: [DISCUSS] Replacing the VR > > > >So based upon this discussion would it be prudent to wait on VyOS 2.0? > The current VR is giving us issues but would the time invested in another > "solution" be wasted especially if by the time another option is chose, > then coded, then tested, then implemented and right as that time happened > to be when VyOS 2.0 is released. Of course you said they are just in the > scoping range so this could still be a year or more out. > > > >Thoughts? > > > >Regards, > >Marty Godsey > >nSource Solutions > > > >-----Original Message----- > >From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On > Behalf Of Will Stevens > >Sent: Friday, September 16, 2016 10:31 AM > >To: dev@cloudstack.apache.org > >Cc: dan...@baturin.org > >Subject: Re: [DISCUSS] Replacing the VR > > > >I just had a quick chat with a couple of the guys over on the VyOS chat. > >I have CC'ed one of them in case we have more licensing questions. > > > >So here is the status with the license "the code inherited from Vyatta > and our modifications from it is GPLv2 (strict, not v2+). The config > reading library is GPLv2 too, so anything that links to is is GPLv2. > >Some auxiliary components we made after the fork are more permissive, > >LGPLv2+ or MIT." > > > >They are currently in the process of scoping a redesign (VyOS 2.0), "we > are planning a clean rewrite that will solve issues of the current config > system". > >This will include the ability to configure via the API. > > > >If we have more questions for VyOS, they are very friendly and > responsive, so we should be able to get answers. > > > >*Will STEVENS* > >Lead Developer > > > >*CloudOps* *| *Cloud Solutions Experts > >420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw > @CloudOps_ > > > >On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sah...@cloudops.com> wrote: > > > >> I agree with Will Ilya. There are so many problems with the VR right > now. > >> Most of the outages we've had recently have somehow involved the VR. > >> We set custom iptables rules on the VR which can and have easily gone > wrong. > >> Openswan is broken, Strongswan replacement still needs to be tested. > >> VVRP with redundant router still needs work, and not to mention the > >> problems we will have when we introduce IPv6 into the whole picture. > >> > >> I think the spirit of the discussion is to rely on a 3rd party to do > >> the networking for us (eg VyOS) and have us handle just the > >> orchestration. All the problems that I've described have already been > >> solved in VyOS. We also get the advantage of a potential wider > >> community to fix and maintain the VR and given our current development > >> velocity, it think it totally makes sense to look for a 3rd party > option. > >> > >> -Syed > >> > >> > >> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <wstev...@cloudops.com> > >> wrote: > >> > >> > The VR has been biting us far too often recently, which is why we > >> > have started looking into alternative implementations. > >> > > >> > One of the things that is nice about potentially using the VyOS is > >> > that > >> it > >> > is based on Debian, so we should be able to run the other services > >> > that > >> we > >> > currently have like the password server and userdata on the VyOS. > >> > This means we would not have to change our architecture initially > >> > and could focus on only replacing the networking paths. > >> > > >> > *Will STEVENS* > >> > Lead Developer > >> > > >> > *CloudOps* *| *Cloud Solutions Experts > >> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* > >> > tw @CloudOps_ > >> > > >> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <n...@li.nux.ro> wrote: > >> > > >> > > The more this is discussed the more I think we should stick with > >> > > our > >> VR. > >> > > > >> > > All these other options either seem unfinished or with > >> > > incompatible license. > >> > > > >> > > VyOS looks the most promising so far, it's a serious, mature > project. > >> > > Adopting it though means we'll have to microservice our way out of > >> > > it > >> > with > >> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve > >> those > >> > > too. Imho this adds complexity we should void. > >> > > > >> > > -- > >> > > Sent from the Delta quadrant using Borg technology! > >> > > > >> > > Nux! > >> > > www.nux.ro > >> > > > >> > > ----- Original Message ----- > >> > > > From: "Will Stevens" <wstev...@cloudops.com> > >> > > > To: dev@cloudstack.apache.org > >> > > > Sent: Thursday, 15 September, 2016 17:21:28 > >> > > > Subject: Re: [DISCUSS] Replacing the VR > >> > > > >> > > > Ya, we would need to add a daemon for VPN as well. Load > >> > > > balancing is another aspect which we will need to consider if we > went this route. > >> > > > Something like https://traefik.io/ could potentially be a good > >> > > > fit > >> due > >> > > to > >> > > > its API driven configuration, but it may be more than what we > need. > >> > > > > >> > > > We should probably try define which pieces make sense to be > >> > > > solved > >> > > together > >> > > > and which pieces would be best suited to be broken out. > >> > > > > >> > > > I think the network connectivity, routing and firewalling should > >> > probably > >> > > > all stay together since the majority of the tools we would > >> potentially > >> > > use > >> > > > would handle all of that together in a single implementation. > >> > > > > >> > > > The password server and userdata seems like a good option for > >> > > > being > >> > > broken > >> > > > out and handled independently (and probably rewritten completely > >> since > >> > > they > >> > > > currently have some issues). > >> > > > > >> > > > Load balancing is another that could warrant splitting out, but > >> > > > that depends on what direction we go and how we would be managing > it. > >> DHCP > >> > > and > >> > > > DNS are others which could go either way. > >> > > > > >> > > > If we do split out services, I think we should consolidate as > >> > > > much as > >> > we > >> > > > can into each service we break out. Ideally a network packet > >> > > > would > >> > never > >> > > > hit more than one, maybe two, services. I don't think we should > >> > > > be splitting services 'just because', I think we need a valid > >> > > > case for splitting any service out because it adds complexity. > >> > > > Our project is already complex enough, we need to avoid adding > >> > > > complexity unless it > >> is > >> > > > really needed. > >> > > > > >> > > > Some more of my thoughts on this anyway... > >> > > > > >> > > > *Will STEVENS* > >> > > > Lead Developer > >> > > > > >> > > > *CloudOps* *| *Cloud Solutions Experts > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com > >> > > > *|* tw @CloudOps_ > >> > > > > >> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <swel...@ena.com> > >> > wrote: > >> > > > > >> > > >> I do agree with you that this probably isn't the right place > >> > > >> the > >> > > password > >> > > >> service and user data. > >> > > >> > >> > > >> > >> > > >> Having said that, after taking a cursory look at the dev docs, > >> > > >> it > >> > > doesn't > >> > > >> seem that difficult to add new daemons: > >> https://opensnaproute.github. > >> > > >> io/docs/developer.html#creating-new-component > >> > > >> > >> > > >> <https://opensnaproute.github.io/docs/developer.html# > >> > > >> creating-new-component> > >> > > >> > >> > > >> > >> > > >> They've definitely build it with a microservices architecture > >> > > >> in > >> mind, > >> > > so > >> > > >> each individual feature is abstracted into it's own small > >> > > >> daemon > >> > > process. > >> > > >> We could just create a daemon for the password server and the > >> userdata > >> > > >> components if we really had to. > >> > > >> > >> > > >> > >> > > >> - Si > >> > > >> > >> > > >> > >> > > >> ________________________________ > >> > > >> From: williamstev...@gmail.com <williamstev...@gmail.com> on > >> > > >> behalf > >> > of > >> > > >> Will Stevens <wstev...@cloudops.com> > >> > > >> Sent: Thursday, September 15, 2016 9:17 AM > >> > > >> To: dev@cloudstack.apache.org > >> > > >> Subject: Re: [DISCUSS] Replacing the VR > >> > > >> > >> > > >> A big part of why I know about it is because it is written in Go. > >> :P > >> > > >> > >> > > >> Yes, it is definitely interesting for the routing and traffic > >> handling > >> > > >> aspects of the VR. We will likely have to rethink some of the > >> pieces > >> > a > >> > > >> little bit like the password server and userdata if we are to > >> > > >> adopt > >> a > >> > > >> different VR approach. This is where I think some of JohnB and > >> > > Chiradeep's > >> > > >> ideas make sense. In many ways, it does not make sense for the > >> device > >> > > >> handling routing and network traffic to also be responsible for > >> > > passwords > >> > > >> and userdata. > >> > > >> > >> > > >> *Will STEVENS* > >> > > >> Lead Developer > >> > > >> > >> > > >> *CloudOps* *| *Cloud Solutions Experts > >> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com > >> > > >> *|* tw @CloudOps_ > >> > > >> > >> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <swel...@ena.com> > >> > wrote: > >> > > >> > >> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks > >> pretty > >> > > >> cool! > >> > > >> > It even supports ONIE install. > >> > > >> > > >> > > >> > To be honest, the ipsec feature could be added, or we could > >> offload > >> > > it to > >> > > >> > separate vm if we needed to. The fact it is so feature rich > >> > > >> > from a > >> > > >> routing > >> > > >> > perspective (and all API driven) is really nice. > >> > > >> > > >> > > >> > > >> > > >> > Based on the roadmap, it looks like they plan to also support > >> > > >> capabilities > >> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This > >> > > >> > will > >> be > >> > > huge > >> > > >> > for our carrier community that rely on these technologies to > >> > > >> > do > >> > > private > >> > > >> > gateway and inter-VPC interconnections today. We handle this > >> > > >> > stuff > >> > on > >> > > our > >> > > >> > ASRs right now with a vlan interconnect into the VR. Being > >> > > >> > able to > >> > do > >> > > >> MPLS > >> > > >> > all the way to the VR would be awesome. > >> > > >> > > >> > > >> > > >> > > >> > It also seems to be written in GO (a language here at ENA we > >> > > >> > know > >> > very > >> > > >> > well). > >> > > >> > > >> > > >> > > >> > > >> > - Si > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > ________________________________ > >> > > >> > From: Will Stevens <williamstev...@gmail.com> > >> > > >> > Sent: Thursday, September 15, 2016 7:06 AM > >> > > >> > To: dev@cloudstack.apache.org > >> > > >> > Subject: RE: [DISCUSS] Replacing the VR > >> > > >> > > >> > > >> > Ya. I don't think it covers our whole use case, but what it > >> > > >> > does > >> > > cover is > >> > > >> > all api driven... > >> > > >> > > >> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> > >> > wrote: > >> > > >> > > >> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it > >> > > >> > > was > >> not > >> > > >> > > intended to do IPSec. > >> > > >> > > > >> > > >> > > It seems as though VyOS is starting to look like the best > >> option. > >> > > >> > > > >> > > >> > > Regards, > >> > > >> > > Marty Godsey > >> > > >> > > nSource Solutions > >> > > >> > > > >> > > >> > > -----Original Message----- > >> > > >> > > From: williamstev...@gmail.com > >> > > >> > > [mailto:williamstev...@gmail.com > >> ] > >> > On > >> > > >> > > Behalf Of Will Stevens > >> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM > >> > > >> > > To: dev@cloudstack.apache.org > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR > >> > > >> > > > >> > > >> > > Or we could go completely crazy and go with something like > >> > > FlexSwitch > >> > > >> > from > >> > > >> > > SnapRoute > >> > > >> > > - http://www.snaproute.com/ > >> > > >> > > - https://opensnaproute.github.io/docs/apis.html > >> > > >> > > > >> > > >> > > *Will STEVENS* > >> > > >> > > Lead Developer > >> > > >> > > > >> > > >> > > *CloudOps* *| *Cloud Solutions Experts > >> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w > >> > > >> > > cloudops.com > >> > *|* > >> > > tw > >> > > >> > > @CloudOps_ > >> > > >> > > > >> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens < > >> > > wstev...@cloudops.com> > >> > > >> > > wrote: > >> > > >> > > > >> > > >> > > > I tend to agree with Syed and Marty. I am not sure what > >> > problems > >> > > are > >> > > >> > > > solved by splitting up the function of the VR into a > >> > > >> > > > bunch of > >> > > >> separate > >> > > >> > > > services. As Syed points out, the complexity added is > >> > > non-trivial. > >> > > >> > > > We now have to manage all the intercontainer networking > >> > > >> > > > as > >> well > >> > as > >> > > >> the > >> > > >> > > > orchestrated ACS networking. > >> > > >> > > > > >> > > >> > > > VyOS is interesting to me because it covers the majority > >> > > >> > > > of > >> our > >> > > use > >> > > >> > > > case with a single unified control plane. It also has > >> > > >> > > > good > >> > > support > >> > > >> > > > for extending features we care about, like IPv6, VXLAN, > >> > > >> > > > VRRP, transactions, etc... > >> > > >> > > > > >> > > >> > > > *Will STEVENS* > >> > > >> > > > Lead Developer > >> > > >> > > > > >> > > >> > > > *CloudOps* *| *Cloud Solutions Experts > >> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w > >> cloudops.com > >> > > *|* > >> > > >> tw > >> > > >> > > > @CloudOps_ > >> > > >> > > > > >> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed < > >> > sah...@cloudops.com> > >> > > >> > wrote: > >> > > >> > > > > >> > > >> > > >> Agree with Marty, adding Docker containers to the > >> > > >> > > >> picture > >> > > although > >> > > >> > > >> can make the VR more flexible but the added complexity > >> > > >> > > >> is > >> just > >> > > not > >> > > >> > > >> worth it. Not to mention we would need to take care of > >> > networking > >> > > >> > > >> each container manually and given that our iptable rules > >> > > >> > > >> are > >> > very > >> > > >> > > >> unstable at the moment I don't see a big value add. > >> > > >> > > >> > >> > > >> > > >> Vyos looks like a better solution to me. I know that it > >> > > >> > > >> does > >> > not > >> > > >> > > >> provide an api but it does fit the bill quite well > >> otherwise. I > >> > > >> > > >> specially like the fact that it has a transaction based > >> > > >> > > >> model > >> > and > >> > > >> you > >> > > >> > > >> can rollback changes if something goes wrong. > >> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey < > >> > > ma...@gonsource.com> > >> > > >> > > wrote: > >> > > >> > > >> > >> > > >> > > >> > Licensing aside, I think splitting the various > >> > > >> > > >> > functions > >> into > >> > > >> > > >> > containers is not a good route either. This will force > >> users > >> > to > >> > > >> > > >> > have to maintain > >> > > >> > > >> and > >> > > >> > > >> > use containers and adds complexity to the networking > >> aspects > >> > of > >> > > >> ACS. > >> > > >> > > >> > Complexity decreases stability. Now I understand the > >> argument > >> > > that > >> > > >> > > >> > a monolithic approach also brings its own set of > >> > > >> > > >> > issues but > >> > it > >> > > >> also > >> > > >> > > >> > simplifies it. > >> > > >> > > >> > > >> > > >> > > >> > Regards, > >> > > >> > > >> > Marty Godsey > >> > > >> > > >> > nSource Solutions > >> > > >> > > >> > > >> > > >> > > >> > -----Original Message----- > >> > > >> > > >> > From: Chiradeep Vittal [mailto:chirade...@gmail.com] > >> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM > >> > > >> > > >> > To: dev@cloudstack.apache.org > >> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR > >> > > >> > > >> > > >> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs > >> > > >> > > >> > of > >> the > >> > > >> > > >> > CloudStack project > >> > > >> > > >> > - it is AGPL licensed. Many enterprises will not > >> > > >> > > >> > touch > >> > > anything > >> > > >> > > >> > that > >> > > >> > > >> has > >> > > >> > > >> > AGPL > >> > > >> > > >> > - the github repo shows rather infrequent updates. > >> > > >> > > >> > Quite > >> > > likely > >> > > >> > > >> > they aren't considering the use cases of the > >> > > >> > > >> > CloudStack > >> > > community > >> > > >> > > >> > > >> > > >> > > >> > I'd back John B's comments on disaggregating the VR. > >> > > >> > > >> > Split > >> it > >> > > into > >> > > >> > > >> > many docker containers > >> > > >> > > >> > - password server > >> > > >> > > >> > - userdata server > >> > > >> > > >> > - DHCP / DNS > >> > > >> > > >> > - s2s VPN > >> > > >> > > >> > - RA VPN > >> > > >> > > >> > - intra-VPC routing and ACL > >> > > >> > > >> > - Port forwarding + NAT > >> > > >> > > >> > - FW > >> > > >> > > >> > - LB (public) > >> > > >> > > >> > - LB (internal), > >> > > >> > > >> > - secondary storage > >> > > >> > > >> > - agent > >> > > >> > > >> > Glue them together with docker compose files (one per > >> > > >> > > >> > use > >> > > case - > >> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc). > >> > > >> > > >> > > >> > > >> > > >> > The VR image then becomes a JeOS + docker. You can > >> > > >> > > >> > test > >> each > >> > of > >> > > >> the > >> > > >> > > >> > components independently and fixing one bug in the > >> > > >> > > >> > field > >> (say > >> > > >> DHCP) > >> > > >> > > >> > is hitless to the other components. You don't need to > >> > > >> > > >> > build per-hypervisor VRs. You could even run on > baremetal. > >> > > >> > > >> > > >> > > >> > > >> > Along the way you need to figure out how to > >> > > >> > > >> > - make the traffic traverse the containers that are > >> > > >> > > >> > needed > >> > to > >> > > be > >> > > >> > > >> > traversed (in most cases just 1) > >> > > >> > > >> > - bootstrap the router (how does it find its compose > file? > >> > > where > >> > > >> > > >> > is the > >> > > >> > > >> > registry?) > >> > > >> > > >> > - rethink the command and control of the VR > >> > > >> > > >> > functions. SSH > >> > > works, > >> > > >> > > >> > but something more declarative, idempotent should be > >> > explored. > >> > > >> > > >> > > >> > > >> > > >> > As you do this, it becomes clearer which of the > >> > > >> > > >> > functions > >> can > >> > > be > >> > > >> > > >> > substituted by for example CloudRouter. Command and > >> > > >> > > >> > Control > >> > of > >> > > the > >> > > >> > > >> docker > >> > > >> > > >> > containers can be moved out to another container. Etc. > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey > >> > > >> > > >> > <ma...@gonsource.com> > >> > > >> > > >> > wrote: > >> > > >> > > >> > > >> > > >> > > >> > > This one does look nice. My biggest concern is the > >> > > >> > > >> > > lack > >> of > >> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned > >> > > >> > > >> > > do not > >> > > have > >> > > >> an > >> > > >> > > >> > > API so we may be stuck at the SSH method. > >> > > >> > > >> > > > >> > > >> > > >> > > Regards, > >> > > >> > > >> > > Marty Godsey > >> > > >> > > >> > > nSource Solutions > >> > > >> > > >> > > > >> > > >> > > >> > > -----Original Message----- > >> > > >> > > >> > > From: Abhinandan Prateek > >> > > >> > > >> > > [mailto:abhinandan.prat...@shapeblue.com] > >> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM > >> > > >> > > >> > > To: dev@cloudstack.apache.org > >> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR > >> > > >> > > >> > > > >> > > >> > > >> > > Cloudrouter looks promising. These have potential to > >> > > >> > > >> > > save > >> > > future > >> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF > etc. > >> > > >> > > >> > > And the best part is they come with test automation > >> > > framework. > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi" > >> > > >> > > >> > > <jayapal.ur...@accelerite.com> > >> > > >> > > >> > > wrote: > >> > > >> > > >> > > > >> > > >> > > >> > > >Hi, > >> > > >> > > >> > > > > >> > > >> > > >> > > >Instead of replacing the VR in first place we > >> > > >> > > >> > > >should add VyOS/cloudrouter > >> > > >> > > >> > > as provider. Once it is stable, network offerings > >> > > >> > > >> > > (on > >> > > upgrade) > >> > > >> > > >> > > can be updated to use it and we can drop the VR if > >> > > >> > > >> > > we > >> want > >> > at > >> > > >> > > >> > > that release > >> > > >> > > >> > onwards. > >> > > >> > > >> > > > > >> > > >> > > >> > > >VR is stabilized over a period of time and some of > >> > > >> > > >> > > >them > >> > are > >> > > >> > > >> > > >running > >> > > >> > > >> > > without issues. When we replicate the ACS VR > >> > > >> > > >> > > features in > >> > new > >> > > >> > > >> > > solution it takes some to find the missing pieces > >> > > >> > > >> > > (hidden > >> > > bugs). > >> > > >> > > >> > > > > >> > > >> > > >> > > >Thanks, > >> > > >> > > >> > > >Jayapal > >> > > >> > > >> > > > > >> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! < > >> > > >> > > >> > > > > >> > > >> > > >> > > >> n...@li.nux.ro> wrote: > >> > > >> > > >> > > >> > >> > > >> > > >> > > >> Hi, > >> > > >> > > >> > > >> > >> > > >> > > >> > > >> I like the idea. > >> > > >> > > >> > > >> > >> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too > >> > > >> > > >> > > >> keen > >> on > >> > > VyOS > >> > > >> > > >> > > >> (it > >> > > >> > > >> > > doesn't have a proper http api etc). > >> > > >> > > >> > > >> > >> > > >> > > >> > > >> -- > >> > > >> > > >> > > >> Sent from the Delta quadrant using Borg technology! > >> > > >> > > >> > > >> > >> > > >> > > >> > > >> Nux! > >> > > >> > > >> > > >> www.nux.ro > >> > > >> > > >> > > >> > >> > > >> > > >> > > >> > >> > > >> > > >> > > abhinandan.prat...@shapeblue.com > >> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com> > >> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > >> > > @shapeblue > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > ----- Original Message ----- > >> > > >> > > >> > > >>> From: "Will Stevens" <williamstev...@gmail.com> > >> > > >> > > >> > > >>> To: dev@cloudstack.apache.org > >> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11 > >> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR > >> > > >> > > >> > > >> > >> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and > >> > > >> > > >> > > >>> should > >> > be > >> > > >> > > >> > > >>> treated as > >> > > >> > > >> > > such. > >> > > >> > > >> > > >>> Please weigh in with the good and bad of this > idea... > >> > > >> > > >> > > >>> > >> > > >> > > >> > > >>> A couple of us have been discussing the idea of > >> > > potentially > >> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open > >> > > >> > > >> > > >>> Source > >> > > Vyatta > >> > > >> > VM). > >> > > >> > > >> > > >>> There may be a license issue because I think it > >> > > >> > > >> > > >>> is > >> > > licensed > >> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's > >> assume > >> > > we > >> > > >> > > >> > > >>> can overcome any > >> > > >> > > >> > > license issues. > >> > > >> > > >> > > >>> > >> > > >> > > >> > > >>> I have spent some time recently with the VyOS > >> > > >> > > >> > > >>> and I > >> > have > >> > > to > >> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and > >> > > intuitive > >> > > >> > > >> > > >>> and it gives you a lot more options for auditing > >> > > >> > > >> > > >>> the > >> > > >> > > configuration etc... > >> > > >> > > >> > > >>> > >> > > >> > > >> > > >>> Items of potential interest: > >> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a > >> simpler > >> > > more > >> > > >> > > >> > > >>> auditable configuration workflow. > >> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support. > >> > > >> > > >> > > >>> - Handles VPN configuration via the same > >> configuration > >> > > >> > > interface. > >> > > >> > > >> > > >>> - Support for OSPF & BGP. > >> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan. > >> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through > >> VRRP. > >> > > >> > > >> > > >>> - VXLAN support. > >> > > >> > > >> > > >>> - Transaction based changes to the VR with > >> > > >> > > >> > > >>> rollback > >> on > >> > > >> error. > >> > > >> > > >> > > >>> > >> > > >> > > >> > > >>> Items that could be difficult to solve: > >> > > >> > > >> > > >>> - Userdata password reset workflow and > >> implementation. > >> > > >> > > >> > > >>> - Upgrade process. > >> > > >> > > >> > > >>> > >> > > >> > > >> > > >>> The VyOS is not the only option if we were to > >> consider > >> > > this > >> > > >> > > >> approach. > >> > > >> > > >> > > >>> Another option, which I don't know as well, > >> > > >> > > >> > > >>> would be CloudRouter (AGPL > >> > > >> > > >> > > >>> license) [2] which is purely API driven. > >> > > >> > > >> > > >>> > >> > > >> > > >> > > >>> Anyway, would love to hear your thoughts... > >> > > >> > > >> > > >>> > >> > > >> > > >> > > >>> Will > >> > > >> > > >> > > >>> > >> > > >> > > >> > > >>> [1] https://vyos.io/ [2] > >> > > >> > > >> > > >>> https://cloudrouter.org/ > >> > > >> > > >> > > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > > >> > > >> > > >> > > >DISCLAIMER > >> > > >> > > >> > > >========== > >> > > >> > > >> > > >This e-mail may contain privileged and confidential > >> > > information > >> > > >> > > >> > > >which is > >> > > >> > > >> > > the property of Accelerite, a Persistent Systems > >> business. > >> > > It is > >> > > >> > > >> > > intended only for the use of the individual or > >> > > >> > > >> > > entity to > >> > > which > >> > > >> it > >> > > >> > > >> > > is addressed. If you are not the intended recipient, > >> > > >> > > >> > > you > >> > are > >> > > not > >> > > >> > > >> > > authorized to read, retain, copy, print, distribute > >> > > >> > > >> > > or > >> use > >> > > this > >> > > >> > > >> > > message. If you have received this communication in > >> error, > >> > > >> please > >> > > >> > > >> > > notify the sender and delete all copies of this > message. > >> > > >> > > >> > > Accelerite, a Persistent Systems business does not > >> > > >> > > >> > > accept > >> > any > >> > > >> > > >> > > liability for virus > >> > > >> > > >> > infected mails. > >> > > >> > > >> > > > >> > > >> > > >> > > >> > > >> > > >> > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > >> > > >> > > >> > > > >> > > >
