Re: [openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Hi, In most cases for plugin developers or fuel users it will be much easier to just write command which he wants to run on nodes instead of describing some abstract task which doesn't have any additional information/logic and looks like unnecessary complexity. But for complicated cases user will have to write some code for tasklib. Thanks, On Wed, Sep 10, 2014 at 8:10 PM, Dmitriy Shulyak dshul...@mirantis.com wrote: Hi, you described transport mechanism for running commands based on facts, we have another one, which stores all business logic in nailgun and only provides orchestrator with set of tasks to execute. This is not a problem. I am talking about API for plugin writer/developer. And how implement it to be more friendly On Wed, Sep 10, 2014 at 6:46 PM, Aleksandr Didenko adide...@mirantis.com wrote: Hi, as for execution of arbitrary code across the OpenStack cluster - I was thinking of mcollective + fact filters: 1) we need to start using mcollective facts [0] [2] - we don't use/configure this currently 2) use mcollective execute_shell_command agent (or any other agent) with fact filter [1] So, for example, if we have mcollective fact called node_roles: node_roles: compute ceph-osd Then we can execute shell cmd on all compute nodes like this: mco rpc execute_shell_command execute cmd=/some_script.sh -F node_role=/compute/ Of course, we can use more complicated filters to run commands more precisely. [0] https://projects.puppetlabs.com/projects/mcollective-plugins/wiki/FactsFacterYAML [1] https://docs.puppetlabs.com/mcollective/reference/ui/filters.html#fact-filters [2] https://docs.puppetlabs.com/mcollective/reference/plugins/facts.html On Wed, Sep 10, 2014 at 6:04 PM, Dmitriy Shulyak dshul...@mirantis.com wrote: Hi folks, Some of you may know that there is ongoing work to achieve kindof data-driven orchestration for Fuel. If this is new to you, please get familiar with spec: https://review.openstack.org/#/c/113491/ Knowing that running random command on nodes will be probably most usable type of orchestration extension, i want to discuss our solution for this problem. Plugin writer will need to do two things: 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is completely configurable, we just need to reach agreement) /etc/puppet/tasks/echo/task.yaml with next content: type: exec cmd: echo 1 2. Provide control plane with orchestration metadata /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 compute: - task: echo description: Simple echo for you priority: 1000 This is done in order to separate concerns of orchestration logic and tasks. From plugin writer perspective it is far more usable to provide exact command in orchestration metadata itself, like: /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec compute: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec I would prefer to stick to the first, because there is benefits of using one interface between all tasks executors (puppet, exec, maybe chef), which will improve debuging and development process. So my question is first - good enough? Or second is essential type of plugin to support? If you want additional implementation details check: https://review.openstack.org/#/c/118311/ https://review.openstack.org/#/c/113226/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Let's not create architectural leaks here. Let there be only tasks, but let's create a really simple template for task that user will be able to easily fill only with the command itself. On Thu, Sep 11, 2014 at 4:17 PM, Evgeniy L e...@mirantis.com wrote: Hi, In most cases for plugin developers or fuel users it will be much easier to just write command which he wants to run on nodes instead of describing some abstract task which doesn't have any additional information/logic and looks like unnecessary complexity. But for complicated cases user will have to write some code for tasklib. Thanks, On Wed, Sep 10, 2014 at 8:10 PM, Dmitriy Shulyak dshul...@mirantis.com wrote: Hi, you described transport mechanism for running commands based on facts, we have another one, which stores all business logic in nailgun and only provides orchestrator with set of tasks to execute. This is not a problem. I am talking about API for plugin writer/developer. And how implement it to be more friendly On Wed, Sep 10, 2014 at 6:46 PM, Aleksandr Didenko adide...@mirantis.com wrote: Hi, as for execution of arbitrary code across the OpenStack cluster - I was thinking of mcollective + fact filters: 1) we need to start using mcollective facts [0] [2] - we don't use/configure this currently 2) use mcollective execute_shell_command agent (or any other agent) with fact filter [1] So, for example, if we have mcollective fact called node_roles: node_roles: compute ceph-osd Then we can execute shell cmd on all compute nodes like this: mco rpc execute_shell_command execute cmd=/some_script.sh -F node_role=/compute/ Of course, we can use more complicated filters to run commands more precisely. [0] https://projects.puppetlabs.com/projects/mcollective-plugins/wiki/FactsFacterYAML [1] https://docs.puppetlabs.com/mcollective/reference/ui/filters.html#fact-filters [2] https://docs.puppetlabs.com/mcollective/reference/plugins/facts.html On Wed, Sep 10, 2014 at 6:04 PM, Dmitriy Shulyak dshul...@mirantis.com wrote: Hi folks, Some of you may know that there is ongoing work to achieve kindof data-driven orchestration for Fuel. If this is new to you, please get familiar with spec: https://review.openstack.org/#/c/113491/ Knowing that running random command on nodes will be probably most usable type of orchestration extension, i want to discuss our solution for this problem. Plugin writer will need to do two things: 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is completely configurable, we just need to reach agreement) /etc/puppet/tasks/echo/task.yaml with next content: type: exec cmd: echo 1 2. Provide control plane with orchestration metadata /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 compute: - task: echo description: Simple echo for you priority: 1000 This is done in order to separate concerns of orchestration logic and tasks. From plugin writer perspective it is far more usable to provide exact command in orchestration metadata itself, like: /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec compute: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec I would prefer to stick to the first, because there is benefits of using one interface between all tasks executors (puppet, exec, maybe chef), which will improve debuging and development process. So my question is first - good enough? Or second is essential type of plugin to support? If you want additional implementation details check: https://review.openstack.org/#/c/118311/ https://review.openstack.org/#/c/113226/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Yours Faithfully, Vladimir Kuklin, Fuel Library Tech Lead, Mirantis, Inc. +7 (495) 640-49-04 +7 (926) 702-39-68 Skype kuklinvv 45bk3, Vorontsovskaya Str. Moscow, Russia, www.mirantis.com http://www.mirantis.ru/ www.mirantis.ru vkuk...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org
[openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Hi folks, Some of you may know that there is ongoing work to achieve kindof data-driven orchestration for Fuel. If this is new to you, please get familiar with spec: https://review.openstack.org/#/c/113491/ Knowing that running random command on nodes will be probably most usable type of orchestration extension, i want to discuss our solution for this problem. Plugin writer will need to do two things: 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is completely configurable, we just need to reach agreement) /etc/puppet/tasks/echo/task.yaml with next content: type: exec cmd: echo 1 2. Provide control plane with orchestration metadata /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 compute: - task: echo description: Simple echo for you priority: 1000 This is done in order to separate concerns of orchestration logic and tasks. From plugin writer perspective it is far more usable to provide exact command in orchestration metadata itself, like: /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec compute: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec I would prefer to stick to the first, because there is benefits of using one interface between all tasks executors (puppet, exec, maybe chef), which will improve debuging and development process. So my question is first - good enough? Or second is essential type of plugin to support? If you want additional implementation details check: https://review.openstack.org/#/c/118311/ https://review.openstack.org/#/c/113226/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Hi, as for execution of arbitrary code across the OpenStack cluster - I was thinking of mcollective + fact filters: 1) we need to start using mcollective facts [0] [2] - we don't use/configure this currently 2) use mcollective execute_shell_command agent (or any other agent) with fact filter [1] So, for example, if we have mcollective fact called node_roles: node_roles: compute ceph-osd Then we can execute shell cmd on all compute nodes like this: mco rpc execute_shell_command execute cmd=/some_script.sh -F node_role=/compute/ Of course, we can use more complicated filters to run commands more precisely. [0] https://projects.puppetlabs.com/projects/mcollective-plugins/wiki/FactsFacterYAML [1] https://docs.puppetlabs.com/mcollective/reference/ui/filters.html#fact-filters [2] https://docs.puppetlabs.com/mcollective/reference/plugins/facts.html On Wed, Sep 10, 2014 at 6:04 PM, Dmitriy Shulyak dshul...@mirantis.com wrote: Hi folks, Some of you may know that there is ongoing work to achieve kindof data-driven orchestration for Fuel. If this is new to you, please get familiar with spec: https://review.openstack.org/#/c/113491/ Knowing that running random command on nodes will be probably most usable type of orchestration extension, i want to discuss our solution for this problem. Plugin writer will need to do two things: 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is completely configurable, we just need to reach agreement) /etc/puppet/tasks/echo/task.yaml with next content: type: exec cmd: echo 1 2. Provide control plane with orchestration metadata /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 compute: - task: echo description: Simple echo for you priority: 1000 This is done in order to separate concerns of orchestration logic and tasks. From plugin writer perspective it is far more usable to provide exact command in orchestration metadata itself, like: /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec compute: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec I would prefer to stick to the first, because there is benefits of using one interface between all tasks executors (puppet, exec, maybe chef), which will improve debuging and development process. So my question is first - good enough? Or second is essential type of plugin to support? If you want additional implementation details check: https://review.openstack.org/#/c/118311/ https://review.openstack.org/#/c/113226/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
