Jira (BOLT-1301) multi-threaded/parallel execution of plans within a plan
Title: Message Title Yasmin Rajabi updated an issue Puppet Task Runner / BOLT-1301 multi-threaded/parallel execution of plans within a plan Change By: Yasmin Rajabi Labels: ghm Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.308327.1557704327000.61709.1566252300296%40Atlassian.JIRA.
Jira (BOLT-1301) multi-threaded/parallel execution of plans within a plan
Title: Message Title Nathan Giuliani commented on BOLT-1301 Re: multi-threaded/parallel execution of plans within a plan Yes, what you've described is how I see it. Using the example above with a plan module::patch_and_reboot, I'd expect to be able to call it from another plan as follows: pan module:patch_lots_of_servers is run on 10 nodes, which does the following: Perform some global pre-patching steps like taking vm snapshots, and emailing users that patching is starting run_plan module::patch_and_reboot on all 10 nodes i'd like puppet to run 10 parallel processes here, each executing the plan on an individual node each node will work through the steps in that plan independently, but execution of this 'top-level' plan won't continue until all 10 parallel processes have been completed Perform some global post-patching steps like removing vm snapshots, and emailing users that patching has finished I did try searching for an existing issue here and didn't come across BOLT-176, but I believe that it is a very similar request, it's just that this one relates to execution of a plan, not a task (which already happens in parallel across nodes based on the concurrency parameter). Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Jira (BOLT-1301) multi-threaded/parallel execution of plans within a plan
Title: Message Title Alex Dreyer commented on BOLT-1301 Re: multi-threaded/parallel execution of plans within a plan We've discussed a few use cases for parallel execution in bolt and come to the conclusion that bolt needs a few different solutions tailored to specific use cases for parallelism. To clarify here the request here is to run the same series of steps across a single set of targets moving on to a subsequent step on one target as soon as the first one is finished? Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.308327.1557704327000.2719.1557771420483%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
Jira (BOLT-1301) multi-threaded/parallel execution of plans within a plan
Title: Message Title Nathan Giuliani created an issue Puppet Task Runner / BOLT-1301 multi-threaded/parallel execution of plans within a plan Issue Type: New Feature Assignee: Unassigned Created: 2019/05/12 4:38 PM Priority: Normal Reporter: Nathan Giuliani Provide the ability to execute plans across multiple nodes in parallel. For example, consider the following scenario that relates to operating system patching: Plan module::patch_and_reboot has the following steps: Perform some pre-patching tasks Install OS updates/patches Reboot the node and wait for it to return Perform some post-patching tasks When executing the above plan on multiple nodes (likely from within from another plan that handles pre/post steps for the entire server list, result handling, etc.), the steps are serialized into a single, synchronous task sequence. If one node takes an hour for step 2 (install updates), but the rest only take 5 minutes, the remaining steps for all other nodes will wait for the single node to complete step 2 before moving on. Workarounds are to code more logic into each of the steps (which are ultimately bolt tasks - scripts), or use an external script to call multiple instances of bolt executing the plan. It would be ideal if this is something bolt could handle itself - either through the ability for run_plan to multi-thread plan execution, or through the implementation of multi-threading/background jobs in a similar fashion to Python threading or PowerShell runspaces.
