Here is a shortened url for the diagram [1] so it won't wrap and be confusing for people...
http://bit.ly/2cQaC0X On Sep 20, 2016 3:51 PM, "Will Stevens" <wstev...@cloudops.com> wrote: > I am going to try to do a bit of a brain dump from me and my team in this > post, and hopefully I can verbalize the ideas well enough that you can all > follow along. Please provide feedback, ideas and suggestions... > > *Preface:* > *--------* > I am currently approaching this problem with the following use cases in > mind. > > - Stabilize/replace the VR for our production cloud deployments as well as > give the community a free, open source and production grade alternative to > the current VR. > > - I have been scoping the integration of the Fortinet Fortigate (hardware > and virtual) into ACS to support the following features. > -- All Isolated Guest Network traffic features (Firewall, PF, NAT, > Client-to-Site VPN, etc) > -- All VPC network traffic features (ACLs, PF, NAT, Client-to-Site and > Site-to-Site VPN, Private Gateway) > -- Public IPv6 support (NAT64 and NAT46) with IPv4 on the guest side. > > *The Setup:* > *----------* > I am not going to get into the details of the Fortinet integration details > in this thread, you can expect a function spec from me soon describing my > plans for this work. I am currently waiting on hardware to validate my > implementation strategy. > > In this post I will be focusing on the way ACS deals with the VR, network > plugins and how this could be potentially adapted to give us more > flexibility going forward. > > I have created a diagram [1] to outline the ideas presented in this post. > > So if you don't like my ideas here, you can simply use the VR as you > currently do and you can basically ignore the dotted circles to the right > of the 'existing VR' in the solid circle. > > The basic idea here, is when you create a network service offering and you > select different Providers to handle different network capabilities, you > can also specify a VM template for those providers (as well as service > offering) which will be used for additional VMs which will be created to > provision those services. In this context, think of templates such as > VyOS, FortiOS, or a VPX to understand the spirit of this approach. > > When a network is provisioned, the current VR will be deployed (ideally > only offering services like; password server, userdata, DHCP, DNS), and in > addition, any network provider templates which have been specified will > also be provisioned. Each 'network VM' will get a leg on both the public > VLAN and the guest VLAN. In the case of a VPC, the first tier will be a > little special (I will get into this in a bit) and will not be able to be > deleted until the VR(s) are all cleaned up and the network is completely > destroyed. This network is denoted as the 'Client MGMT' network in the > diagram [1]. > > You will notice that ONLY the 'existing VR' has network connectivity to > the ACS MGMT server. It may not necessarily be through a 169.x.x.x > address, but that illustrates the point for now. The reason for this is > because we can not guarantee that a Guest User does may not gain access to > these VRs, and in cases I will get into later, the Guest User may actually > be given access to those VRs. This does however mean that the ACS MGMT > server does not have direct access to those VRs in order to configure > them. To solve this, the 'existing VR' would act as a proxy for the ACS > Management server over the ACS MGMT network to pass configuration commands > to the VRs to the right of the 'existing VR' depicted in the graphic. > > *The Details:* > *------------* > Now that you have a basic idea of the deployment structure I have in mind, > let's discuss some of the finer details. > > - We should be able to relatively easily adapt existing external network > plugins to be able to be provisioned with this mechanism (assuming they > have a VM option). The Palo Alto would be an example which I would likely > use as a proof of concept for this adaptation. It would be important that > the external plugin could work as an external physical appliance OR as a > dynamically provisioned VM appliance. > > - Enterprise grade network appliances which support VM deployment will no > longer have to be pre-deployed and shared, but instead will be able to be > dynamically provisioned on a network by network basis. > > - Existing external network appliance plugins targeting physical boxes > will still work in concert with this approach without any need for > modification. > > - We could add an 'Allow Guest Access' checkbox to the service provider > section in the network service offering to give the Guest User access to > their network appliance. This is especially useful for an implementation > like the FortiGate because it allows for the user to configure UTM features > which ACS does not orchestrate but the appliance is capable of. This also > applies to devices like the NetScaler. In the diagram [1] this is depicted > by the dashed green lines between the devices and the 'Client MGMT' > network. So in the case of the NetScaler, you would get an NSIP on the > Client MGMT network which would allow the Guest User to connect to that > box. I envision a URL pointing to the guest network IP with > username/password returned for a network which a user can use to connect > into that network appliance. > > - Similar to above, the Guest User will have much more visibility into the > logging and access on the network. I can't count how many times a customer > has asked for an easy way to get the IPsec logs from the VR. > > I think this is enough to get the idea across. I have worked through a > bunch of examples on a white board with my team to try to work through a > bunch of the use cases, edge cases and deployment scenarios, but I am sure > there are situations we have not thought through yet. > > The current VR has been a constant pain for us and our service customers, > so it is likely that we will be taking some action one way or another > because this needs to be addressed. I know there is a lot going on here, > but hopefully I have been able to get our ideas across. > > Please ask questions or make suggestions. Are there major pieces we are > not taking into account? Do you have use cases which would be simplified > by this approach that I have not called out? > > Would love to hear your feedback... > > [1] https://objects-east.cloud.ca/v1/5ef827605f884961b94881e > 928e7a250/swill/vr_network.png > > *Will STEVENS* > Lead Developer > > *CloudOps* *| *Cloud Solutions Experts > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 > w cloudops.com *|* tw @CloudOps_ > > On Tue, Sep 20, 2016 at 5:54 AM, Murali Reddy <muralimmre...@gmail.com> > wrote: > >> Configuration management of network appliances particularly for Cloud and >> NFV scenarios is still evolving area. Programmability is the not the >> strength for even the most popular network operating systems like IOS, >> JunoS etc. So its not surprising why CloudStack integrates in a archaic way >> with stock linux for the VR. >> >> VR was never integral/hardcoded option in CloudStack. Its always been a >> plugin. CloudStack network orchestration is well abstracted and designed >> with vision to compose a network with different set of providers for >> different services. Yes that vision is not fully realised yet, and we don’t >> have true service function chains. That would be different discussion topic. >> >> I tend to agree with Simon, as alternate/interim option we can take hard >> look whats causing the problems with current VR integration. Personally, I >> think it would be easier we take a cue from configuration managers and >> network configuration solutions out there (for e.g promise theory based >> Cisco ACI) move to more declarative model of expressing desired state of >> network configuration. Infact current VR from 4.6, actually holds the >> desired state per service basis, seems to be in that direction. >> >> It does make sense to evaluate new appliances which can provide rich >> semantics (like programmable API, declarative configuration, versioning, >> commit/rollback etc), but will need significant engineering effort and time >> to stabilise. We may make same mistakes with integration of other appliance >> as well, if we fully don’t understand the nature of the current problems >> with CloudStack core and service provider interaction and current VR >> integration. >> >> >> On 16/09/16, 11:59 PM, "Simon Weller" <swel...@ena.com> wrote: >> >> >I think our other option is to take a real look at what it would take to >> fix the VR. In my opinion, a lot of the problems are related to the >> monolithic python code base and the fact nothing is actually separated. >> > >> >Secondly, the python scripts (and bash scripts) don't use any >> established libraries to complete tasks and instead shell out and run >> commands that are both hard to track and hard to parse on return. >> > >> > >> >If we daemonized this, used a real api for Agent to VR communication, >> used common already existing libraries for the system service and network >> interactions and spent a bit of time separating out code into distinct >> modules, everything would behave a lot better. >> > >> > >> >The pain and suffering is due to years and years of patches and constant >> shelling out to complete tasks in my opinion. If we spend time to rethink >> how we interact with the VR in general and we abstract the systems and >> networking stuff and use well known and stable libraries to do the work, >> the VR would be much easier to maintain. >> > >> > >> >- Si >> > >> > >> > >> > >> >________________________________ >> >From: Marty Godsey <ma...@gonsource.com> >> >Sent: Friday, September 16, 2016 12:24 PM >> >To: dev@cloudstack.apache.org >> >Subject: RE: [DISCUSS] Replacing the VR >> > >> >So based upon this discussion would it be prudent to wait on VyOS 2.0? >> The current VR is giving us issues but would the time invested in another >> "solution" be wasted especially if by the time another option is chose, >> then coded, then tested, then implemented and right as that time happened >> to be when VyOS 2.0 is released. Of course you said they are just in the >> scoping range so this could still be a year or more out. >> > >> >Thoughts? >> > >> >Regards, >> >Marty Godsey >> >nSource Solutions >> > >> >-----Original Message----- >> >From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On >> Behalf Of Will Stevens >> >Sent: Friday, September 16, 2016 10:31 AM >> >To: dev@cloudstack.apache.org >> >Cc: dan...@baturin.org >> >Subject: Re: [DISCUSS] Replacing the VR >> > >> >I just had a quick chat with a couple of the guys over on the VyOS chat. >> >I have CC'ed one of them in case we have more licensing questions. >> > >> >So here is the status with the license "the code inherited from Vyatta >> and our modifications from it is GPLv2 (strict, not v2+). The config >> reading library is GPLv2 too, so anything that links to is is GPLv2. >> >Some auxiliary components we made after the fork are more permissive, >> >LGPLv2+ or MIT." >> > >> >They are currently in the process of scoping a redesign (VyOS 2.0), "we >> are planning a clean rewrite that will solve issues of the current config >> system". >> >This will include the ability to configure via the API. >> > >> >If we have more questions for VyOS, they are very friendly and >> responsive, so we should be able to get answers. >> > >> >*Will STEVENS* >> >Lead Developer >> > >> >*CloudOps* *| *Cloud Solutions Experts >> >420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw >> @CloudOps_ >> > >> >On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sah...@cloudops.com> wrote: >> > >> >> I agree with Will Ilya. There are so many problems with the VR right >> now. >> >> Most of the outages we've had recently have somehow involved the VR. >> >> We set custom iptables rules on the VR which can and have easily gone >> wrong. >> >> Openswan is broken, Strongswan replacement still needs to be tested. >> >> VVRP with redundant router still needs work, and not to mention the >> >> problems we will have when we introduce IPv6 into the whole picture. >> >> >> >> I think the spirit of the discussion is to rely on a 3rd party to do >> >> the networking for us (eg VyOS) and have us handle just the >> >> orchestration. All the problems that I've described have already been >> >> solved in VyOS. We also get the advantage of a potential wider >> >> community to fix and maintain the VR and given our current development >> >> velocity, it think it totally makes sense to look for a 3rd party >> option. >> >> >> >> -Syed >> >> >> >> >> >> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <wstev...@cloudops.com> >> >> wrote: >> >> >> >> > The VR has been biting us far too often recently, which is why we >> >> > have started looking into alternative implementations. >> >> > >> >> > One of the things that is nice about potentially using the VyOS is >> >> > that >> >> it >> >> > is based on Debian, so we should be able to run the other services >> >> > that >> >> we >> >> > currently have like the password server and userdata on the VyOS. >> >> > This means we would not have to change our architecture initially >> >> > and could focus on only replacing the networking paths. >> >> > >> >> > *Will STEVENS* >> >> > Lead Developer >> >> > >> >> > *CloudOps* *| *Cloud Solutions Experts >> >> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* >> >> > tw @CloudOps_ >> >> > >> >> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <n...@li.nux.ro> wrote: >> >> > >> >> > > The more this is discussed the more I think we should stick with >> >> > > our >> >> VR. >> >> > > >> >> > > All these other options either seem unfinished or with >> >> > > incompatible license. >> >> > > >> >> > > VyOS looks the most promising so far, it's a serious, mature >> project. >> >> > > Adopting it though means we'll have to microservice our way out of >> >> > > it >> >> > with >> >> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve >> >> those >> >> > > too. Imho this adds complexity we should void. >> >> > > >> >> > > -- >> >> > > Sent from the Delta quadrant using Borg technology! >> >> > > >> >> > > Nux! >> >> > > www.nux.ro >> >> > > >> >> > > ----- Original Message ----- >> >> > > > From: "Will Stevens" <wstev...@cloudops.com> >> >> > > > To: dev@cloudstack.apache.org >> >> > > > Sent: Thursday, 15 September, 2016 17:21:28 >> >> > > > Subject: Re: [DISCUSS] Replacing the VR >> >> > > >> >> > > > Ya, we would need to add a daemon for VPN as well. Load >> >> > > > balancing is another aspect which we will need to consider if we >> went this route. >> >> > > > Something like https://traefik.io/ could potentially be a good >> >> > > > fit >> >> due >> >> > > to >> >> > > > its API driven configuration, but it may be more than what we >> need. >> >> > > > >> >> > > > We should probably try define which pieces make sense to be >> >> > > > solved >> >> > > together >> >> > > > and which pieces would be best suited to be broken out. >> >> > > > >> >> > > > I think the network connectivity, routing and firewalling should >> >> > probably >> >> > > > all stay together since the majority of the tools we would >> >> potentially >> >> > > use >> >> > > > would handle all of that together in a single implementation. >> >> > > > >> >> > > > The password server and userdata seems like a good option for >> >> > > > being >> >> > > broken >> >> > > > out and handled independently (and probably rewritten completely >> >> since >> >> > > they >> >> > > > currently have some issues). >> >> > > > >> >> > > > Load balancing is another that could warrant splitting out, but >> >> > > > that depends on what direction we go and how we would be >> managing it. >> >> DHCP >> >> > > and >> >> > > > DNS are others which could go either way. >> >> > > > >> >> > > > If we do split out services, I think we should consolidate as >> >> > > > much as >> >> > we >> >> > > > can into each service we break out. Ideally a network packet >> >> > > > would >> >> > never >> >> > > > hit more than one, maybe two, services. I don't think we should >> >> > > > be splitting services 'just because', I think we need a valid >> >> > > > case for splitting any service out because it adds complexity. >> >> > > > Our project is already complex enough, we need to avoid adding >> >> > > > complexity unless it >> >> is >> >> > > > really needed. >> >> > > > >> >> > > > Some more of my thoughts on this anyway... >> >> > > > >> >> > > > *Will STEVENS* >> >> > > > Lead Developer >> >> > > > >> >> > > > *CloudOps* *| *Cloud Solutions Experts >> >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com >> >> > > > *|* tw @CloudOps_ >> >> > > > >> >> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <swel...@ena.com> >> >> > wrote: >> >> > > > >> >> > > >> I do agree with you that this probably isn't the right place >> >> > > >> the >> >> > > password >> >> > > >> service and user data. >> >> > > >> >> >> > > >> >> >> > > >> Having said that, after taking a cursory look at the dev docs, >> >> > > >> it >> >> > > doesn't >> >> > > >> seem that difficult to add new daemons: >> >> https://opensnaproute.github. >> >> > > >> io/docs/developer.html#creating-new-component >> >> > > >> >> >> > > >> <https://opensnaproute.github.io/docs/developer.html# >> >> > > >> creating-new-component> >> >> > > >> >> >> > > >> >> >> > > >> They've definitely build it with a microservices architecture >> >> > > >> in >> >> mind, >> >> > > so >> >> > > >> each individual feature is abstracted into it's own small >> >> > > >> daemon >> >> > > process. >> >> > > >> We could just create a daemon for the password server and the >> >> userdata >> >> > > >> components if we really had to. >> >> > > >> >> >> > > >> >> >> > > >> - Si >> >> > > >> >> >> > > >> >> >> > > >> ________________________________ >> >> > > >> From: williamstev...@gmail.com <williamstev...@gmail.com> on >> >> > > >> behalf >> >> > of >> >> > > >> Will Stevens <wstev...@cloudops.com> >> >> > > >> Sent: Thursday, September 15, 2016 9:17 AM >> >> > > >> To: dev@cloudstack.apache.org >> >> > > >> Subject: Re: [DISCUSS] Replacing the VR >> >> > > >> >> >> > > >> A big part of why I know about it is because it is written in >> Go. >> >> :P >> >> > > >> >> >> > > >> Yes, it is definitely interesting for the routing and traffic >> >> handling >> >> > > >> aspects of the VR. We will likely have to rethink some of the >> >> pieces >> >> > a >> >> > > >> little bit like the password server and userdata if we are to >> >> > > >> adopt >> >> a >> >> > > >> different VR approach. This is where I think some of JohnB and >> >> > > Chiradeep's >> >> > > >> ideas make sense. In many ways, it does not make sense for the >> >> device >> >> > > >> handling routing and network traffic to also be responsible for >> >> > > passwords >> >> > > >> and userdata. >> >> > > >> >> >> > > >> *Will STEVENS* >> >> > > >> Lead Developer >> >> > > >> >> >> > > >> *CloudOps* *| *Cloud Solutions Experts >> >> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com >> >> > > >> *|* tw @CloudOps_ >> >> > > >> >> >> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <swel...@ena.com> >> >> > wrote: >> >> > > >> >> >> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks >> >> pretty >> >> > > >> cool! >> >> > > >> > It even supports ONIE install. >> >> > > >> > >> >> > > >> > To be honest, the ipsec feature could be added, or we could >> >> offload >> >> > > it to >> >> > > >> > separate vm if we needed to. The fact it is so feature rich >> >> > > >> > from a >> >> > > >> routing >> >> > > >> > perspective (and all API driven) is really nice. >> >> > > >> > >> >> > > >> > >> >> > > >> > Based on the roadmap, it looks like they plan to also support >> >> > > >> capabilities >> >> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This >> >> > > >> > will >> >> be >> >> > > huge >> >> > > >> > for our carrier community that rely on these technologies to >> >> > > >> > do >> >> > > private >> >> > > >> > gateway and inter-VPC interconnections today. We handle this >> >> > > >> > stuff >> >> > on >> >> > > our >> >> > > >> > ASRs right now with a vlan interconnect into the VR. Being >> >> > > >> > able to >> >> > do >> >> > > >> MPLS >> >> > > >> > all the way to the VR would be awesome. >> >> > > >> > >> >> > > >> > >> >> > > >> > It also seems to be written in GO (a language here at ENA we >> >> > > >> > know >> >> > very >> >> > > >> > well). >> >> > > >> > >> >> > > >> > >> >> > > >> > - Si >> >> > > >> > >> >> > > >> > >> >> > > >> > >> >> > > >> > ________________________________ >> >> > > >> > From: Will Stevens <williamstev...@gmail.com> >> >> > > >> > Sent: Thursday, September 15, 2016 7:06 AM >> >> > > >> > To: dev@cloudstack.apache.org >> >> > > >> > Subject: RE: [DISCUSS] Replacing the VR >> >> > > >> > >> >> > > >> > Ya. I don't think it covers our whole use case, but what it >> >> > > >> > does >> >> > > cover is >> >> > > >> > all api driven... >> >> > > >> > >> >> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> >> >> > wrote: >> >> > > >> > >> >> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it >> >> > > >> > > was >> >> not >> >> > > >> > > intended to do IPSec. >> >> > > >> > > >> >> > > >> > > It seems as though VyOS is starting to look like the best >> >> option. >> >> > > >> > > >> >> > > >> > > Regards, >> >> > > >> > > Marty Godsey >> >> > > >> > > nSource Solutions >> >> > > >> > > >> >> > > >> > > -----Original Message----- >> >> > > >> > > From: williamstev...@gmail.com >> >> > > >> > > [mailto:williamstev...@gmail.com >> >> ] >> >> > On >> >> > > >> > > Behalf Of Will Stevens >> >> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM >> >> > > >> > > To: dev@cloudstack.apache.org >> >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR >> >> > > >> > > >> >> > > >> > > Or we could go completely crazy and go with something like >> >> > > FlexSwitch >> >> > > >> > from >> >> > > >> > > SnapRoute >> >> > > >> > > - http://www.snaproute.com/ >> >> > > >> > > - https://opensnaproute.github.io/docs/apis.html >> >> > > >> > > >> >> > > >> > > *Will STEVENS* >> >> > > >> > > Lead Developer >> >> > > >> > > >> >> > > >> > > *CloudOps* *| *Cloud Solutions Experts >> >> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w >> >> > > >> > > cloudops.com >> >> > *|* >> >> > > tw >> >> > > >> > > @CloudOps_ >> >> > > >> > > >> >> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens < >> >> > > wstev...@cloudops.com> >> >> > > >> > > wrote: >> >> > > >> > > >> >> > > >> > > > I tend to agree with Syed and Marty. I am not sure what >> >> > problems >> >> > > are >> >> > > >> > > > solved by splitting up the function of the VR into a >> >> > > >> > > > bunch of >> >> > > >> separate >> >> > > >> > > > services. As Syed points out, the complexity added is >> >> > > non-trivial. >> >> > > >> > > > We now have to manage all the intercontainer networking >> >> > > >> > > > as >> >> well >> >> > as >> >> > > >> the >> >> > > >> > > > orchestrated ACS networking. >> >> > > >> > > > >> >> > > >> > > > VyOS is interesting to me because it covers the majority >> >> > > >> > > > of >> >> our >> >> > > use >> >> > > >> > > > case with a single unified control plane. It also has >> >> > > >> > > > good >> >> > > support >> >> > > >> > > > for extending features we care about, like IPv6, VXLAN, >> >> > > >> > > > VRRP, transactions, etc... >> >> > > >> > > > >> >> > > >> > > > *Will STEVENS* >> >> > > >> > > > Lead Developer >> >> > > >> > > > >> >> > > >> > > > *CloudOps* *| *Cloud Solutions Experts >> >> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w >> >> cloudops.com >> >> > > *|* >> >> > > >> tw >> >> > > >> > > > @CloudOps_ >> >> > > >> > > > >> >> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed < >> >> > sah...@cloudops.com> >> >> > > >> > wrote: >> >> > > >> > > > >> >> > > >> > > >> Agree with Marty, adding Docker containers to the >> >> > > >> > > >> picture >> >> > > although >> >> > > >> > > >> can make the VR more flexible but the added complexity >> >> > > >> > > >> is >> >> just >> >> > > not >> >> > > >> > > >> worth it. Not to mention we would need to take care of >> >> > networking >> >> > > >> > > >> each container manually and given that our iptable rules >> >> > > >> > > >> are >> >> > very >> >> > > >> > > >> unstable at the moment I don't see a big value add. >> >> > > >> > > >> >> >> > > >> > > >> Vyos looks like a better solution to me. I know that it >> >> > > >> > > >> does >> >> > not >> >> > > >> > > >> provide an api but it does fit the bill quite well >> >> otherwise. I >> >> > > >> > > >> specially like the fact that it has a transaction based >> >> > > >> > > >> model >> >> > and >> >> > > >> you >> >> > > >> > > >> can rollback changes if something goes wrong. >> >> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey < >> >> > > ma...@gonsource.com> >> >> > > >> > > wrote: >> >> > > >> > > >> >> >> > > >> > > >> > Licensing aside, I think splitting the various >> >> > > >> > > >> > functions >> >> into >> >> > > >> > > >> > containers is not a good route either. This will force >> >> users >> >> > to >> >> > > >> > > >> > have to maintain >> >> > > >> > > >> and >> >> > > >> > > >> > use containers and adds complexity to the networking >> >> aspects >> >> > of >> >> > > >> ACS. >> >> > > >> > > >> > Complexity decreases stability. Now I understand the >> >> argument >> >> > > that >> >> > > >> > > >> > a monolithic approach also brings its own set of >> >> > > >> > > >> > issues but >> >> > it >> >> > > >> also >> >> > > >> > > >> > simplifies it. >> >> > > >> > > >> > >> >> > > >> > > >> > Regards, >> >> > > >> > > >> > Marty Godsey >> >> > > >> > > >> > nSource Solutions >> >> > > >> > > >> > >> >> > > >> > > >> > -----Original Message----- >> >> > > >> > > >> > From: Chiradeep Vittal [mailto:chirade...@gmail.com] >> >> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM >> >> > > >> > > >> > To: dev@cloudstack.apache.org >> >> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR >> >> > > >> > > >> > >> >> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs >> >> > > >> > > >> > of >> >> the >> >> > > >> > > >> > CloudStack project >> >> > > >> > > >> > - it is AGPL licensed. Many enterprises will not >> >> > > >> > > >> > touch >> >> > > anything >> >> > > >> > > >> > that >> >> > > >> > > >> has >> >> > > >> > > >> > AGPL >> >> > > >> > > >> > - the github repo shows rather infrequent updates. >> >> > > >> > > >> > Quite >> >> > > likely >> >> > > >> > > >> > they aren't considering the use cases of the >> >> > > >> > > >> > CloudStack >> >> > > community >> >> > > >> > > >> > >> >> > > >> > > >> > I'd back John B's comments on disaggregating the VR. >> >> > > >> > > >> > Split >> >> it >> >> > > into >> >> > > >> > > >> > many docker containers >> >> > > >> > > >> > - password server >> >> > > >> > > >> > - userdata server >> >> > > >> > > >> > - DHCP / DNS >> >> > > >> > > >> > - s2s VPN >> >> > > >> > > >> > - RA VPN >> >> > > >> > > >> > - intra-VPC routing and ACL >> >> > > >> > > >> > - Port forwarding + NAT >> >> > > >> > > >> > - FW >> >> > > >> > > >> > - LB (public) >> >> > > >> > > >> > - LB (internal), >> >> > > >> > > >> > - secondary storage >> >> > > >> > > >> > - agent >> >> > > >> > > >> > Glue them together with docker compose files (one per >> >> > > >> > > >> > use >> >> > > case - >> >> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc). >> >> > > >> > > >> > >> >> > > >> > > >> > The VR image then becomes a JeOS + docker. You can >> >> > > >> > > >> > test >> >> each >> >> > of >> >> > > >> the >> >> > > >> > > >> > components independently and fixing one bug in the >> >> > > >> > > >> > field >> >> (say >> >> > > >> DHCP) >> >> > > >> > > >> > is hitless to the other components. You don't need to >> >> > > >> > > >> > build per-hypervisor VRs. You could even run on >> baremetal. >> >> > > >> > > >> > >> >> > > >> > > >> > Along the way you need to figure out how to >> >> > > >> > > >> > - make the traffic traverse the containers that are >> >> > > >> > > >> > needed >> >> > to >> >> > > be >> >> > > >> > > >> > traversed (in most cases just 1) >> >> > > >> > > >> > - bootstrap the router (how does it find its compose >> file? >> >> > > where >> >> > > >> > > >> > is the >> >> > > >> > > >> > registry?) >> >> > > >> > > >> > - rethink the command and control of the VR >> >> > > >> > > >> > functions. SSH >> >> > > works, >> >> > > >> > > >> > but something more declarative, idempotent should be >> >> > explored. >> >> > > >> > > >> > >> >> > > >> > > >> > As you do this, it becomes clearer which of the >> >> > > >> > > >> > functions >> >> can >> >> > > be >> >> > > >> > > >> > substituted by for example CloudRouter. Command and >> >> > > >> > > >> > Control >> >> > of >> >> > > the >> >> > > >> > > >> docker >> >> > > >> > > >> > containers can be moved out to another container. Etc. >> >> > > >> > > >> > >> >> > > >> > > >> > >> >> > > >> > > >> > >> >> > > >> > > >> > >> >> > > >> > > >> > >> >> > > >> > > >> > >> >> > > >> > > >> > >> >> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey >> >> > > >> > > >> > <ma...@gonsource.com> >> >> > > >> > > >> > wrote: >> >> > > >> > > >> > >> >> > > >> > > >> > > This one does look nice. My biggest concern is the >> >> > > >> > > >> > > lack >> >> of >> >> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned >> >> > > >> > > >> > > do not >> >> > > have >> >> > > >> an >> >> > > >> > > >> > > API so we may be stuck at the SSH method. >> >> > > >> > > >> > > >> >> > > >> > > >> > > Regards, >> >> > > >> > > >> > > Marty Godsey >> >> > > >> > > >> > > nSource Solutions >> >> > > >> > > >> > > >> >> > > >> > > >> > > -----Original Message----- >> >> > > >> > > >> > > From: Abhinandan Prateek >> >> > > >> > > >> > > [mailto:abhinandan.prat...@shapeblue.com] >> >> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM >> >> > > >> > > >> > > To: dev@cloudstack.apache.org >> >> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR >> >> > > >> > > >> > > >> >> > > >> > > >> > > Cloudrouter looks promising. These have potential to >> >> > > >> > > >> > > save >> >> > > future >> >> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF >> etc. >> >> > > >> > > >> > > And the best part is they come with test automation >> >> > > framework. >> >> > > >> > > >> > > >> >> > > >> > > >> > > >> >> > > >> > > >> > > >> >> > > >> > > >> > > >> >> > > >> > > >> > > >> >> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi" >> >> > > >> > > >> > > <jayapal.ur...@accelerite.com> >> >> > > >> > > >> > > wrote: >> >> > > >> > > >> > > >> >> > > >> > > >> > > >Hi, >> >> > > >> > > >> > > > >> >> > > >> > > >> > > >Instead of replacing the VR in first place we >> >> > > >> > > >> > > >should add VyOS/cloudrouter >> >> > > >> > > >> > > as provider. Once it is stable, network offerings >> >> > > >> > > >> > > (on >> >> > > upgrade) >> >> > > >> > > >> > > can be updated to use it and we can drop the VR if >> >> > > >> > > >> > > we >> >> want >> >> > at >> >> > > >> > > >> > > that release >> >> > > >> > > >> > onwards. >> >> > > >> > > >> > > > >> >> > > >> > > >> > > >VR is stabilized over a period of time and some of >> >> > > >> > > >> > > >them >> >> > are >> >> > > >> > > >> > > >running >> >> > > >> > > >> > > without issues. When we replicate the ACS VR >> >> > > >> > > >> > > features in >> >> > new >> >> > > >> > > >> > > solution it takes some to find the missing pieces >> >> > > >> > > >> > > (hidden >> >> > > bugs). >> >> > > >> > > >> > > > >> >> > > >> > > >> > > >Thanks, >> >> > > >> > > >> > > >Jayapal >> >> > > >> > > >> > > > >> >> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! < >> >> > > >> > > >> > > > >> >> > > >> > > >> > > >> n...@li.nux.ro> wrote: >> >> > > >> > > >> > > >> >> >> > > >> > > >> > > >> Hi, >> >> > > >> > > >> > > >> >> >> > > >> > > >> > > >> I like the idea. >> >> > > >> > > >> > > >> >> >> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too >> >> > > >> > > >> > > >> keen >> >> on >> >> > > VyOS >> >> > > >> > > >> > > >> (it >> >> > > >> > > >> > > doesn't have a proper http api etc). >> >> > > >> > > >> > > >> >> >> > > >> > > >> > > >> -- >> >> > > >> > > >> > > >> Sent from the Delta quadrant using Borg >> technology! >> >> > > >> > > >> > > >> >> >> > > >> > > >> > > >> Nux! >> >> > > >> > > >> > > >> www.nux.ro >> >> > > >> > > >> > > >> >> >> > > >> > > >> > > >> >> >> > > >> > > >> > > abhinandan.prat...@shapeblue.com >> >> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com> >> >> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK >> >> > > @shapeblue >> >> > > >> > > >> > > >> >> > > >> > > >> > > >> >> > > >> > > >> > > >> >> > > >> > > >> > > ----- Original Message ----- >> >> > > >> > > >> > > >>> From: "Will Stevens" <williamstev...@gmail.com> >> >> > > >> > > >> > > >>> To: dev@cloudstack.apache.org >> >> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11 >> >> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR >> >> > > >> > > >> > > >> >> >> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and >> >> > > >> > > >> > > >>> should >> >> > be >> >> > > >> > > >> > > >>> treated as >> >> > > >> > > >> > > such. >> >> > > >> > > >> > > >>> Please weigh in with the good and bad of this >> idea... >> >> > > >> > > >> > > >>> >> >> > > >> > > >> > > >>> A couple of us have been discussing the idea of >> >> > > potentially >> >> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open >> >> > > >> > > >> > > >>> Source >> >> > > Vyatta >> >> > > >> > VM). >> >> > > >> > > >> > > >>> There may be a license issue because I think it >> >> > > >> > > >> > > >>> is >> >> > > licensed >> >> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's >> >> assume >> >> > > we >> >> > > >> > > >> > > >>> can overcome any >> >> > > >> > > >> > > license issues. >> >> > > >> > > >> > > >>> >> >> > > >> > > >> > > >>> I have spent some time recently with the VyOS >> >> > > >> > > >> > > >>> and I >> >> > have >> >> > > to >> >> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and >> >> > > intuitive >> >> > > >> > > >> > > >>> and it gives you a lot more options for auditing >> >> > > >> > > >> > > >>> the >> >> > > >> > > configuration etc... >> >> > > >> > > >> > > >>> >> >> > > >> > > >> > > >>> Items of potential interest: >> >> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a >> >> simpler >> >> > > more >> >> > > >> > > >> > > >>> auditable configuration workflow. >> >> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support. >> >> > > >> > > >> > > >>> - Handles VPN configuration via the same >> >> configuration >> >> > > >> > > interface. >> >> > > >> > > >> > > >>> - Support for OSPF & BGP. >> >> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan. >> >> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through >> >> VRRP. >> >> > > >> > > >> > > >>> - VXLAN support. >> >> > > >> > > >> > > >>> - Transaction based changes to the VR with >> >> > > >> > > >> > > >>> rollback >> >> on >> >> > > >> error. >> >> > > >> > > >> > > >>> >> >> > > >> > > >> > > >>> Items that could be difficult to solve: >> >> > > >> > > >> > > >>> - Userdata password reset workflow and >> >> implementation. >> >> > > >> > > >> > > >>> - Upgrade process. >> >> > > >> > > >> > > >>> >> >> > > >> > > >> > > >>> The VyOS is not the only option if we were to >> >> consider >> >> > > this >> >> > > >> > > >> approach. >> >> > > >> > > >> > > >>> Another option, which I don't know as well, >> >> > > >> > > >> > > >>> would be CloudRouter (AGPL >> >> > > >> > > >> > > >>> license) [2] which is purely API driven. >> >> > > >> > > >> > > >>> >> >> > > >> > > >> > > >>> Anyway, would love to hear your thoughts... >> >> > > >> > > >> > > >>> >> >> > > >> > > >> > > >>> Will >> >> > > >> > > >> > > >>> >> >> > > >> > > >> > > >>> [1] https://vyos.io/ [2] >> >> > > >> > > >> > > >>> https://cloudrouter.org/ >> >> > > >> > > >> > > > >> >> > > >> > > >> > > > >> >> > > >> > > >> > > > >> >> > > >> > > >> > > > >> >> > > >> > > >> > > >DISCLAIMER >> >> > > >> > > >> > > >========== >> >> > > >> > > >> > > >This e-mail may contain privileged and confidential >> >> > > information >> >> > > >> > > >> > > >which is >> >> > > >> > > >> > > the property of Accelerite, a Persistent Systems >> >> business. >> >> > > It is >> >> > > >> > > >> > > intended only for the use of the individual or >> >> > > >> > > >> > > entity to >> >> > > which >> >> > > >> it >> >> > > >> > > >> > > is addressed. If you are not the intended recipient, >> >> > > >> > > >> > > you >> >> > are >> >> > > not >> >> > > >> > > >> > > authorized to read, retain, copy, print, distribute >> >> > > >> > > >> > > or >> >> use >> >> > > this >> >> > > >> > > >> > > message. If you have received this communication in >> >> error, >> >> > > >> please >> >> > > >> > > >> > > notify the sender and delete all copies of this >> message. >> >> > > >> > > >> > > Accelerite, a Persistent Systems business does not >> >> > > >> > > >> > > accept >> >> > any >> >> > > >> > > >> > > liability for virus >> >> > > >> > > >> > infected mails. >> >> > > >> > > >> > > >> >> > > >> > > >> > >> >> > > >> > > >> >> >> > > >> > > > >> >> > > >> > > > >> >> > > >> > > >> >> > > >> > >> >> > > >> >> > >> >> >
