Hi Adrian Sounds like there's a need for a true workflow engine here. I suspect that for any use case beyond trivial, each user will have her own flavor of CD (TripleO is a good example for this) - some will prefer to use a canary server and test it manually before deploying to other servers, some will deploy to specific servers based on the subset of users they're serving, some will do a red/black type of deployment (Which maps well into environments below), etc. But I think that there should be an user-configurable mechanism to define and tweak these workflows. There will also be additional related workflows - e.g. rolling back a deployment, or updating the infrastructure components of a running application (e.g. database version upgrade). I might be raising a topic that's already been discussed (sorry if I am), but I was wondering how this would fit into the architecture.
Thanks! Uri On Oct 31, 2013, at 6:42 AM, Adrian Otto <adrian.o...@rackspace.com> wrote: > Robert, > > We would be interested in exploring your use case for CD, and help you judge > what would be the best fit. I know that Monty Taylor has identified some > parts of openstack-infra that we could potentially leverage in Solum to > address our CI/CD goals. The design of that feature is still pending, so we > are looking forward to sourcing its of input on that. > > I don't anticipate any fundament gaps that would prevent you from using a > future version of Solum to help narrow your scope. The whole point of our > focus is to try to make these goals easier to achieve, and try to get maximum > leverage of what the OpenStack ecosystem already offers. > > The high level concept behind the GIt Integration feature currently conceived > for Solum is to offer application developers a way to commit code to a Git > repo and upon a git push command, have a remote system trigger a modular > CI/CD pipeline that ends with that application running on an OpenStack cloud > in one or more "environments" (dev/test/stage/prod/etc.). In a simple case > that may be (in the case of a dynamic language like Python) the testing of > the code, arrangement of the commit into a deployable bundle, triggering of > the Solum API to trigger a deployment, automatic generation of a HOT > artifact, and stimulation of the Heat API and any other API calls needed. > > In a more complex scenario, you may want additional features and > capabilities, such as a system like Gerrit to trigger the commit/push, > integrate an artifact repository for storing the builds, etc. Using Zuul for > that would be appropriate for sure. > > There are also dev shops who already have a lot invested into their Jenkins > setups, and will want to bypass portions of what I described above, so there > will be a use case supported where you can just integrate with the solum API, > and pass in a pre-built bundle artifact and bypass part or all of Solum's CI > features, and just use it for deployment and re-deployment and promotion of a > deployment between various environments (dev/test/pros/etc.). You would > expect features like canary deploy in that case that might make it more > attractive than just generating your own HOT artifact in your Jenkins > workflow and passing that in directly for orchestration. > > You should expect a higher level of control for management of the deployed > system with abstracts that let you easily extend the system for monitoring > and management reasons. > > Note that not all this functionality belongs in Solum. Some of it may > actually come from Heat, and other downstream systems. The idea is that we > address the key gaps. > > Adrian > > On Oct 30, 2013, at 9:16 PM, Robert Collins <robe...@robertcollins.net> > wrote: > >> Hi Adrian, >> I'm very interested in Solum, particularly as it relates to >> TripleO : as you know the TripleO story is to treat OpenStack as just >> another application to install via Heat/Nova etc - and we've got a >> major story around pulling from git -> CI + CD pipeline -> automated >> deployment to production. >> >> So - where we can say 'hey, this should be part of Solum', we can hive >> stuff out of the TripleO program into Solum - but only if we've got >> compatible stories. >> >> Right now, our design for CD is basically Gerrit + Zuul - exactly the >> same as gating checks for OpenStack itself. When I look at >> https://wiki.openstack.org/wiki/Solum/FeatureBlueprints/GitIntegration#Flow_Boundaries >> I'm having a little trouble mapping that into your design. >> >> I'm wondering if you can expand on how that hangs together? >> >> Thanks! >> Rob >> >> On 31 October 2013 14:01, Adrian Otto <adrian.o...@rackspace.com> wrote: >>> Clayton, >>> >>> Thanks for adding the diagram illustrating the flow. I expect that "respond >>> to client" may not be a synchronous flow, rather that if creation of a repo >>> takes a while that the API may return a 202 Accepted, and the client can >>> poll a status attribute to determine completion and learn the actual >>> location of the repo created. So in that case "respond to client" also >>> might mean "update async status". This will be particularly important in >>> cases where an external SCM system is used (perhaps Github). >>> >>> I agree that this will be a helpful reference to guide our upcoming >>> interactive design sessions. I have created the first call for >>> participation, which will also be separately announced: >>> >>> https://wiki.openstack.org/wiki/Solum/BreakoutMeetings >>> >>> Adrian >>> >>> On Oct 30, 2013, at 2:45 PM, Clayton Coleman <ccole...@redhat.com> wrote: >>> >>>> In the IRC meeting yesterday [1] we discussed splitting the individual >>>> topics for the git deploy blueprint [2] into rough sub areas. To help >>>> frame the abstractions we've been discussing I roughed out a flow based on >>>> two user inputs, REST API create and a git push and then tried to draw >>>> boxes around the related bits. The abstractions roughly correspond to the >>>> potential areas that can be the focus of break out discussions (and if >>>> they don't a good discussion of why is always helpful). >>>> >>>> Squiggly boxes are potential plugin points and/or strong responsibility >>>> boundaries. Feedback desirable. >>>> >>>> [1] http://irclogs.solum.io/2013/solum.2013-10-29-16.01.txt >>>> [2] >>>> https://wiki.openstack.org/wiki/Solum/FeatureBlueprints/GitIntegration#Flow_Boundaries >>>> >>>> >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> -- >> Robert Collins <rbtcoll...@hp.com> >> Distinguished Technologist >> HP Converged Cloud >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
