+100500, decoupling our Puppet modules would simplify a lot of things. A few comments inline:
On Thu, Jan 16, 2014 at 3:43 AM, Dmitry Ilyin <[email protected]> wrote: > So we have an idea to separate this manifest to many small ones. Each of > them would do only one task and orchestration engine would apply them in > required order > to deploy the node user have requested. With such architecture each 'stage' > would be more or less isolated and it would be possible for a single > developer > to add a new 'stage' that would install his 'plugin'. Considering that "stage" has a very specific meaning in Puppet, reusing the same word here would introduce unnecessary confusion. Unless you actually plan to use Puppet stages mechanism to implement this approach (afaiu you intend to use orchestrator for that instead) I think we should use a different term, e.g. "component" or "task". > * Even if we'll make each 'stage' a separate manifest there still we be a > need to use our current puppet modules as library. Or we should make every > modules fully self-contained? Or we could use something like python's > virtualenv to provide each 'stage' with correct set of library modules? In the spirit of the primary principle of refactoring (making one major change at a time), I'd start with keeping our current modules library intact. First refactoring should be replacing cross-module dependencies with hard separation of groups of modules into stages/components/tasks. Once that is working reasonably well, then we can consider rearranging the library (which is also a long-overdue refactoring, but large enough to be done as a separate step). > * What is the best way to install packages then? Keep our current repository > on master node or invent somethin else? I think current repository should be fine. The only piece missing is tools to add more packages into the RPM and DEB repositories on the master node. A cleaner but more complex option would be to create additional repositories on master nodes, and updating apt/yum configs on target nodes to use all such repositories during deployment. The latter would also make it easier to add a mechanism to distribute security updates at some later point. > * How and where to store information on what 'stages' should be done for > which roles? There should be some metadata with fixed scenarios or there > should be dynamic graph? I would combine this problem with the need for an "empty" role. For each role, define a full ordered list of stages (components/tasks) to be applied. Then extract all common tasks shared between all roles into the empty role and make that role a dependency of all other roles. > * Some 'plugins' and 'stages' may require to add options to settings panel. > If we are going to have external plugins there should be a way to hook int > setting and add your options. > This data will the be available in astute.yaml. I think we should implement ability to override parts of astute.yaml in plugins: allow plugin developers to provide yaml files that replace parts of the astute.yaml structure or add new sections without redefining the rest of the data. Here's how I implemented that in one of my personal projects: https://github.com/angdraug/samizdat/blob/master/lib/samizdat/engine/site.rb#L111 > * Our orchestation engine will have to become much more complex then what we > have now. There should be queues of stages for each deploying node. And > there should be support > for inter-node dependencies of stages. Is Heat going to make this problem easier to solve? -- Dmitry Borodaenko -- Mailing list: https://launchpad.net/~fuel-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~fuel-dev More help : https://help.launchpad.net/ListHelp
