Re: [openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Let's not create architectural leaks here. Let there be only tasks, but let's create a really simple template for task that user will be able to easily fill only with the command itself. On Thu, Sep 11, 2014 at 4:17 PM, Evgeniy L wrote: > Hi, > > In most cases for plugin developers or fuel users it will be much > easier to just write command which he wants to run on nodes > instead of describing some abstract task which doesn't have > any additional information/logic and looks like unnecessary complexity. > > But for complicated cases user will have to write some code for tasklib. > > Thanks, > > On Wed, Sep 10, 2014 at 8:10 PM, Dmitriy Shulyak > wrote: > >> Hi, >> >> you described transport mechanism for running commands based on facts, we >> have another one, which stores >> all business logic in nailgun and only provides orchestrator with set of >> tasks to execute. This is not a problem. >> >> I am talking about API for plugin writer/developer. And how implement it >> to be more "friendly" >> >> On Wed, Sep 10, 2014 at 6:46 PM, Aleksandr Didenko > > wrote: >> >>> Hi, >>> >>> as for execution of arbitrary code across the OpenStack cluster - I was >>> thinking of mcollective + fact filters: >>> >>> 1) we need to start using mcollective facts [0] [2] - we don't >>> use/configure this currently >>> 2) use mcollective execute_shell_command agent (or any other agent) with >>> fact filter [1] >>> >>> So, for example, if we have mcollective fact called "node_roles": >>> node_roles: "compute ceph-osd" >>> >>> Then we can execute shell cmd on all compute nodes like this: >>> >>> mco rpc execute_shell_command execute cmd="/some_script.sh" -F >>> "node_role=/compute/" >>> >>> Of course, we can use more complicated filters to run commands more >>> precisely. >>> >>> [0] >>> https://projects.puppetlabs.com/projects/mcollective-plugins/wiki/FactsFacterYAML >>> [1] >>> https://docs.puppetlabs.com/mcollective/reference/ui/filters.html#fact-filters >>> [2] https://docs.puppetlabs.com/mcollective/reference/plugins/facts.html >>> >>> >>> On Wed, Sep 10, 2014 at 6:04 PM, Dmitriy Shulyak >>> wrote: >>> Hi folks, Some of you may know that there is ongoing work to achieve kindof data-driven orchestration for Fuel. If this is new to you, please get familiar with spec: https://review.openstack.org/#/c/113491/ Knowing that running random command on nodes will be probably most usable type of orchestration extension, i want to discuss our solution for this problem. Plugin writer will need to do two things: 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is completely configurable, we just need to reach agreement) /etc/puppet/tasks/echo/task.yaml with next content: type: exec cmd: echo 1 2. Provide control plane with orchestration metadata /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 compute: - task: echo description: Simple echo for you priority: 1000 This is done in order to separate concerns of orchestration logic and tasks. From plugin writer perspective it is far more usable to provide exact command in orchestration metadata itself, like: /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec compute: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec I would prefer to stick to the first, because there is benefits of using one interface between all tasks executors (puppet, exec, maybe chef), which will improve debuging and development process. So my question is first - good enough? Or second is essential type of plugin to support? If you want additional implementation details check: https://review.openstack.org/#/c/118311/ https://review.openstack.org/#/c/113226/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/m
Re: [openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Hi, In most cases for plugin developers or fuel users it will be much easier to just write command which he wants to run on nodes instead of describing some abstract task which doesn't have any additional information/logic and looks like unnecessary complexity. But for complicated cases user will have to write some code for tasklib. Thanks, On Wed, Sep 10, 2014 at 8:10 PM, Dmitriy Shulyak wrote: > Hi, > > you described transport mechanism for running commands based on facts, we > have another one, which stores > all business logic in nailgun and only provides orchestrator with set of > tasks to execute. This is not a problem. > > I am talking about API for plugin writer/developer. And how implement it > to be more "friendly" > > On Wed, Sep 10, 2014 at 6:46 PM, Aleksandr Didenko > wrote: > >> Hi, >> >> as for execution of arbitrary code across the OpenStack cluster - I was >> thinking of mcollective + fact filters: >> >> 1) we need to start using mcollective facts [0] [2] - we don't >> use/configure this currently >> 2) use mcollective execute_shell_command agent (or any other agent) with >> fact filter [1] >> >> So, for example, if we have mcollective fact called "node_roles": >> node_roles: "compute ceph-osd" >> >> Then we can execute shell cmd on all compute nodes like this: >> >> mco rpc execute_shell_command execute cmd="/some_script.sh" -F >> "node_role=/compute/" >> >> Of course, we can use more complicated filters to run commands more >> precisely. >> >> [0] >> https://projects.puppetlabs.com/projects/mcollective-plugins/wiki/FactsFacterYAML >> [1] >> https://docs.puppetlabs.com/mcollective/reference/ui/filters.html#fact-filters >> [2] https://docs.puppetlabs.com/mcollective/reference/plugins/facts.html >> >> >> On Wed, Sep 10, 2014 at 6:04 PM, Dmitriy Shulyak >> wrote: >> >>> Hi folks, >>> >>> Some of you may know that there is ongoing work to achieve kindof >>> data-driven orchestration >>> for Fuel. If this is new to you, please get familiar with spec: >>> >>> https://review.openstack.org/#/c/113491/ >>> >>> Knowing that running random command on nodes will be probably most >>> usable type of >>> orchestration extension, i want to discuss our solution for this problem. >>> >>> Plugin writer will need to do two things: >>> >>> 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is >>> completely configurable, >>> we just need to reach agreement) >>> >>> /etc/puppet/tasks/echo/task.yaml >>> >>> with next content: >>> >>>type: exec >>>cmd: echo 1 >>> >>> 2. Provide control plane with orchestration metadata >>> >>> /etc/fuel/tasks/echo_task.yaml >>> >>> controller: >>> - >>> task: echo >>> description: Simple echo for you >>> priority: 1000 >>> compute: >>> - >>> task: echo >>> description: Simple echo for you >>> priority: 1000 >>> >>> This is done in order to separate concerns of orchestration logic and >>> tasks. >>> >>> From plugin writer perspective it is far more usable to provide exact >>> command in orchestration metadata itself, like: >>> >>> /etc/fuel/tasks/echo_task.yaml >>> >>> controller: >>> - >>> task: echo >>> description: Simple echo for you >>> priority: 1000 >>> cmd: echo 1 >>> type: exec >>> >>> compute: >>> - >>> task: echo >>> description: Simple echo for you >>> priority: 1000 >>> cmd: echo 1 >>> type: exec >>> >>> I would prefer to stick to the first, because there is benefits of using >>> one interface between all tasks executors (puppet, exec, maybe chef), which >>> will improve debuging and development process. >>> >>> So my question is first - good enough? Or second is essential type of >>> plugin to support? >>> >>> If you want additional implementation details check: >>> https://review.openstack.org/#/c/118311/ >>> https://review.openstack.org/#/c/113226/ >>> >>> >>> >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Hi, you described transport mechanism for running commands based on facts, we have another one, which stores all business logic in nailgun and only provides orchestrator with set of tasks to execute. This is not a problem. I am talking about API for plugin writer/developer. And how implement it to be more "friendly" On Wed, Sep 10, 2014 at 6:46 PM, Aleksandr Didenko wrote: > Hi, > > as for execution of arbitrary code across the OpenStack cluster - I was > thinking of mcollective + fact filters: > > 1) we need to start using mcollective facts [0] [2] - we don't > use/configure this currently > 2) use mcollective execute_shell_command agent (or any other agent) with > fact filter [1] > > So, for example, if we have mcollective fact called "node_roles": > node_roles: "compute ceph-osd" > > Then we can execute shell cmd on all compute nodes like this: > > mco rpc execute_shell_command execute cmd="/some_script.sh" -F > "node_role=/compute/" > > Of course, we can use more complicated filters to run commands more > precisely. > > [0] > https://projects.puppetlabs.com/projects/mcollective-plugins/wiki/FactsFacterYAML > [1] > https://docs.puppetlabs.com/mcollective/reference/ui/filters.html#fact-filters > [2] https://docs.puppetlabs.com/mcollective/reference/plugins/facts.html > > > On Wed, Sep 10, 2014 at 6:04 PM, Dmitriy Shulyak > wrote: > >> Hi folks, >> >> Some of you may know that there is ongoing work to achieve kindof >> data-driven orchestration >> for Fuel. If this is new to you, please get familiar with spec: >> >> https://review.openstack.org/#/c/113491/ >> >> Knowing that running random command on nodes will be probably most usable >> type of >> orchestration extension, i want to discuss our solution for this problem. >> >> Plugin writer will need to do two things: >> >> 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is >> completely configurable, >> we just need to reach agreement) >> >> /etc/puppet/tasks/echo/task.yaml >> >> with next content: >> >>type: exec >>cmd: echo 1 >> >> 2. Provide control plane with orchestration metadata >> >> /etc/fuel/tasks/echo_task.yaml >> >> controller: >> - >> task: echo >> description: Simple echo for you >> priority: 1000 >> compute: >> - >> task: echo >> description: Simple echo for you >> priority: 1000 >> >> This is done in order to separate concerns of orchestration logic and >> tasks. >> >> From plugin writer perspective it is far more usable to provide exact >> command in orchestration metadata itself, like: >> >> /etc/fuel/tasks/echo_task.yaml >> >> controller: >> - >> task: echo >> description: Simple echo for you >> priority: 1000 >> cmd: echo 1 >> type: exec >> >> compute: >> - >> task: echo >> description: Simple echo for you >> priority: 1000 >> cmd: echo 1 >> type: exec >> >> I would prefer to stick to the first, because there is benefits of using >> one interface between all tasks executors (puppet, exec, maybe chef), which >> will improve debuging and development process. >> >> So my question is first - good enough? Or second is essential type of >> plugin to support? >> >> If you want additional implementation details check: >> https://review.openstack.org/#/c/118311/ >> https://review.openstack.org/#/c/113226/ >> >> >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Hi, as for execution of arbitrary code across the OpenStack cluster - I was thinking of mcollective + fact filters: 1) we need to start using mcollective facts [0] [2] - we don't use/configure this currently 2) use mcollective execute_shell_command agent (or any other agent) with fact filter [1] So, for example, if we have mcollective fact called "node_roles": node_roles: "compute ceph-osd" Then we can execute shell cmd on all compute nodes like this: mco rpc execute_shell_command execute cmd="/some_script.sh" -F "node_role=/compute/" Of course, we can use more complicated filters to run commands more precisely. [0] https://projects.puppetlabs.com/projects/mcollective-plugins/wiki/FactsFacterYAML [1] https://docs.puppetlabs.com/mcollective/reference/ui/filters.html#fact-filters [2] https://docs.puppetlabs.com/mcollective/reference/plugins/facts.html On Wed, Sep 10, 2014 at 6:04 PM, Dmitriy Shulyak wrote: > Hi folks, > > Some of you may know that there is ongoing work to achieve kindof > data-driven orchestration > for Fuel. If this is new to you, please get familiar with spec: > > https://review.openstack.org/#/c/113491/ > > Knowing that running random command on nodes will be probably most usable > type of > orchestration extension, i want to discuss our solution for this problem. > > Plugin writer will need to do two things: > > 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is > completely configurable, > we just need to reach agreement) > > /etc/puppet/tasks/echo/task.yaml > > with next content: > >type: exec >cmd: echo 1 > > 2. Provide control plane with orchestration metadata > > /etc/fuel/tasks/echo_task.yaml > > controller: > - > task: echo > description: Simple echo for you > priority: 1000 > compute: > - > task: echo > description: Simple echo for you > priority: 1000 > > This is done in order to separate concerns of orchestration logic and > tasks. > > From plugin writer perspective it is far more usable to provide exact > command in orchestration metadata itself, like: > > /etc/fuel/tasks/echo_task.yaml > > controller: > - > task: echo > description: Simple echo for you > priority: 1000 > cmd: echo 1 > type: exec > > compute: > - > task: echo > description: Simple echo for you > priority: 1000 > cmd: echo 1 > type: exec > > I would prefer to stick to the first, because there is benefits of using > one interface between all tasks executors (puppet, exec, maybe chef), which > will improve debuging and development process. > > So my question is first - good enough? Or second is essential type of > plugin to support? > > If you want additional implementation details check: > https://review.openstack.org/#/c/118311/ > https://review.openstack.org/#/c/113226/ > > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
