Re: [capistrano-mailing-list] Apply parallelism (concurrency) to multiple tasks

[Eric Rutherford]

My "mixed results" was completely a misunderstanding on my part.  I didn't 
realize you needed to re-enable a task that's invoked in a loop, otherwise 
it's just called once.  Thanks for your help, I've figured out what I was 
doing wrong.

Eric

On Friday, March 18, 2016 at 9:04:34 AM UTC-5, Lee Hambley wrote:
>
> I thought I could invoke a task from the on() block and have it execute a 
>> task/function, but I had mixed results when I did that.  I also wasn't sure 
>> of that was really how that was intended to be used.
>
>
> For whatever it's worth you can write your code in classes, and then just 
> pass the SSHKit context into them:
>
>
>- 
>https://github.com/capistrano/capistrano/blob/master/lib/capistrano/git.rb
>- 
>
> https://github.com/capistrano/capistrano/blob/master/lib/capistrano/tasks/git.rake#L3
>  
>(note passing `self`)
>
>
> This would allow you to do some testing, however
>
> The on() block is the unit of synchronisation, one on() block will wait 
>> for all servers to complete before it returns.
>
>  -- SSHKit README.md 
>> 
>
>
> If you can send me some pseudo code of what you wanted to work, I'd be 
> more than glad to try and go a bit deeper, it's also worth pointing out 
> that there's a pretty decent split between the task structure 
> (`namespace`/`task` from Rake, which makes up the Capistrano DSL) and the 
> underlying *do something on a server* code (SSHKit) ... the latter should 
> be threadsafe as the parallel() helpers in SSHKit make liberal use of 
> threads...
>
> What I'm trying to say is:
>
>- You can:
>   - Write your own classes (and, test them!)
>   - Avoid our synchronisation primitives and use threads if ours 
>   don't work
>
> I hope that helps, but I must admit that a handful of lines of pseudo code 
> from your end would raise my confidence factor in making suggestions 10x.
>
> Cheers, Lee
>
>
> Lee Hambley
> http://lee.hambley.name/
> +49 (0) 170 298 5667
>
> On 18 March 2016 at 04:48, Eric Rutherford  > wrote:
>
>> Hello Lee
>> No problem at all on the length of time for the response (I actually 
>> considered it to be pretty quick) and I appreciate your time on the 
>> matter.  
>>
>> I think the problem may be that the tasks I'm trying to group together 
>> are ones that are executed on the host and others that are executed by the 
>> capistrano run.  For example I've built a capistrano plugin for 
>> de-registering and registering our nodes with our AWS ELBs.  I'd like to 
>> call that deregistration task on the same node that I'm restarting the 
>> service for, then call the registration task after the restart before 
>> de-registration/restart task on the next node.
>>
>> I thought I could invoke a task from the on() block and have it execute a 
>> task/function, but I had mixed results when I did that.  I also wasn't sure 
>> of that was really how that was intended to be used.
>>
>> Any hints or a kick in the right direction would be greatly appreciated.
>>
>> Eric
>>
>> On Thursday, March 17, 2016 at 1:46:00 AM UTC-5, Lee Hambley wrote:
>>>
>>> HI Eric,
>>>
>>> Certainly if you're looking for restricting, or choosing your preferred 
>>> kind of parallelism/concurrency for your own tasks you are in the right 
>>> place with the `on(in: )` functions (
>>> https://github.com/capistrano/sshkit/blob/master/README.md#parallel)
>>>
>>> For Capistrano and it's plugins you are at the mercy of the maintainers 
>>> (yours truly) or the plugin authors to have exposed the concurrency 
>>> settings.
>>>
>>> If there's any more I can help with (long question, short answer!) I'm 
>>> more than happy, just drop me a note, you shouldn't have to wait 20h in the 
>>> moderation queue whilst I ignore your email next time ;-)
>>>
>>> Lee Hambley
>>> http://lee.hambley.name/
>>> +49 (0) 170 298 5667
>>>
>>> On 16 March 2016 at 19:24, Eric Rutherford  wrote:
>>>
 Versions:

- Ruby 2.0.0
- Capistrano 3.4.0

 I'm trying to work out a solution to limit the number of servers that a 
 group of tasks (remove from load balancer, push code, restart service, add 
 back to load balancer for example) can run on at a time.  I've been 
 digging 
 through past conversations and documentation, but I've failed to find a 
 solution with capistrano 3.x so far.  I've found some examples from v2.x, 
 but most of the work in that example relied on features that are no longer 
 in existence.  I've been working through creating a custom deploy task 
 that 
 specifies "in: :sequence" and calls outside tasks, but I'm not sure if I'm 
 on the right track and figured that I might ask for some pointers (or a 
 push in the right direction) before I head too far in the wrong direction.

 -- 
 You received this message because you are subscribed to 

Re: [capistrano-mailing-list] Apply parallelism (concurrency) to multiple tasks

[Eric Rutherford]

Hello Lee
No problem at all on the length of time for the response (I actually 
considered it to be pretty quick) and I appreciate your time on the 
matter.  

I think the problem may be that the tasks I'm trying to group together are 
ones that are executed on the host and others that are executed by the 
capistrano run.  For example I've built a capistrano plugin for 
de-registering and registering our nodes with our AWS ELBs.  I'd like to 
call that deregistration task on the same node that I'm restarting the 
service for, then call the registration task after the restart before 
de-registration/restart task on the next node.

I thought I could invoke a task from the on() block and have it execute a 
task/function, but I had mixed results when I did that.  I also wasn't sure 
of that was really how that was intended to be used.

Any hints or a kick in the right direction would be greatly appreciated.

Eric

On Thursday, March 17, 2016 at 1:46:00 AM UTC-5, Lee Hambley wrote:
>
> HI Eric,
>
> Certainly if you're looking for restricting, or choosing your preferred 
> kind of parallelism/concurrency for your own tasks you are in the right 
> place with the `on(in: )` functions (
> https://github.com/capistrano/sshkit/blob/master/README.md#parallel)
>
> For Capistrano and it's plugins you are at the mercy of the maintainers 
> (yours truly) or the plugin authors to have exposed the concurrency 
> settings.
>
> If there's any more I can help with (long question, short answer!) I'm 
> more than happy, just drop me a note, you shouldn't have to wait 20h in the 
> moderation queue whilst I ignore your email next time ;-)
>
> Lee Hambley
> http://lee.hambley.name/
> +49 (0) 170 298 5667
>
> On 16 March 2016 at 19:24, Eric Rutherford  > wrote:
>
>> Versions:
>>
>>- Ruby 2.0.0
>>- Capistrano 3.4.0
>>
>> I'm trying to work out a solution to limit the number of servers that a 
>> group of tasks (remove from load balancer, push code, restart service, add 
>> back to load balancer for example) can run on at a time.  I've been digging 
>> through past conversations and documentation, but I've failed to find a 
>> solution with capistrano 3.x so far.  I've found some examples from v2.x, 
>> but most of the work in that example relied on features that are no longer 
>> in existence.  I've been working through creating a custom deploy task that 
>> specifies "in: :sequence" and calls outside tasks, but I'm not sure if I'm 
>> on the right track and figured that I might ask for some pointers (or a 
>> push in the right direction) before I head too far in the wrong direction.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Capistrano" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to capistrano+...@googlegroups.com .
>> To view this discussion on the web, visit 
>> https://groups.google.com/d/msgid/capistrano/cf49591c-0be0-44d6-b1b3-c44bf2027a91%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capistrano+unsubscr...@googlegroups.com.
To view this discussion on the web, visit 
https://groups.google.com/d/msgid/capistrano/d71a508b-e9f5-44f8-a5e0-510b28f6a517%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [capistrano-mailing-list] Apply parallelism (concurrency) to multiple tasks

[Lee Hambley]

http://blog.simplificator.com/2014/12/30/rake-execute-vs-invoke/

Might help, although honestly if you get as far as a Cap/Rake task like
"rolling restart of servers", it makes sense to just pass the context
object into class and manage that yourself.

Honestly, I think invoke vs. execute catches people out frequently enough
that I ought to alias one to the other. Rake as a build tool, invoke is
more useful for avoiding unnecessary work, for tools like Cap, I barely see
the case for having a "once" option.

Lee Hambley
http://lee.hambley.name/
+49 (0) 170 298 5667

On 18 March 2016 at 22:06, Eric Rutherford  wrote:

> My "mixed results" was completely a misunderstanding on my part.  I didn't
> realize you needed to re-enable a task that's invoked in a loop, otherwise
> it's just called once.  Thanks for your help, I've figured out what I was
> doing wrong.
>
> Eric
>
> On Friday, March 18, 2016 at 9:04:34 AM UTC-5, Lee Hambley wrote:
>>
>> I thought I could invoke a task from the on() block and have it execute a
>>> task/function, but I had mixed results when I did that.  I also wasn't sure
>>> of that was really how that was intended to be used.
>>
>>
>> For whatever it's worth you can write your code in classes, and then just
>> pass the SSHKit context into them:
>>
>>
>>-
>>https://github.com/capistrano/capistrano/blob/master/lib/capistrano/git.rb
>>-
>>
>> https://github.com/capistrano/capistrano/blob/master/lib/capistrano/tasks/git.rake#L3
>>(note passing `self`)
>>
>>
>> This would allow you to do some testing, however
>>
>> The on() block is the unit of synchronisation, one on() block will wait
>>> for all servers to complete before it returns.
>>
>>  -- SSHKit README.md
>>> 
>>
>>
>> If you can send me some pseudo code of what you wanted to work, I'd be
>> more than glad to try and go a bit deeper, it's also worth pointing out
>> that there's a pretty decent split between the task structure
>> (`namespace`/`task` from Rake, which makes up the Capistrano DSL) and the
>> underlying *do something on a server* code (SSHKit) ... the latter
>> should be threadsafe as the parallel() helpers in SSHKit make liberal use
>> of threads...
>>
>> What I'm trying to say is:
>>
>>- You can:
>>   - Write your own classes (and, test them!)
>>   - Avoid our synchronisation primitives and use threads if ours
>>   don't work
>>
>> I hope that helps, but I must admit that a handful of lines of pseudo
>> code from your end would raise my confidence factor in making suggestions
>> 10x.
>>
>> Cheers, Lee
>>
>>
>> Lee Hambley
>> http://lee.hambley.name/
>> +49 (0) 170 298 5667
>>
>> On 18 March 2016 at 04:48, Eric Rutherford  wrote:
>>
>>> Hello Lee
>>> No problem at all on the length of time for the response (I actually
>>> considered it to be pretty quick) and I appreciate your time on the
>>> matter.
>>>
>>> I think the problem may be that the tasks I'm trying to group together
>>> are ones that are executed on the host and others that are executed by the
>>> capistrano run.  For example I've built a capistrano plugin for
>>> de-registering and registering our nodes with our AWS ELBs.  I'd like to
>>> call that deregistration task on the same node that I'm restarting the
>>> service for, then call the registration task after the restart before
>>> de-registration/restart task on the next node.
>>>
>>> I thought I could invoke a task from the on() block and have it execute
>>> a task/function, but I had mixed results when I did that.  I also wasn't
>>> sure of that was really how that was intended to be used.
>>>
>>> Any hints or a kick in the right direction would be greatly appreciated.
>>>
>>> Eric
>>>
>>> On Thursday, March 17, 2016 at 1:46:00 AM UTC-5, Lee Hambley wrote:

 HI Eric,

 Certainly if you're looking for restricting, or choosing your preferred
 kind of parallelism/concurrency for your own tasks you are in the right
 place with the `on(in: )` functions (
 https://github.com/capistrano/sshkit/blob/master/README.md#parallel)

 For Capistrano and it's plugins you are at the mercy of the maintainers
 (yours truly) or the plugin authors to have exposed the concurrency
 settings.

 If there's any more I can help with (long question, short answer!) I'm
 more than happy, just drop me a note, you shouldn't have to wait 20h in the
 moderation queue whilst I ignore your email next time ;-)

 Lee Hambley
 http://lee.hambley.name/
 +49 (0) 170 298 5667

 On 16 March 2016 at 19:24, Eric Rutherford  wrote:

> Versions:
>
>- Ruby 2.0.0
>- Capistrano 3.4.0
>
> I'm trying to work out a solution to limit the number of servers that
> a group of tasks (remove from load balancer, push code, restart service,
> add back to 

Re: [capistrano-mailing-list] Apply parallelism (concurrency) to multiple tasks

[Lee Hambley]

HI Eric,

Certainly if you're looking for restricting, or choosing your preferred
kind of parallelism/concurrency for your own tasks you are in the right
place with the `on(in: )` functions (
https://github.com/capistrano/sshkit/blob/master/README.md#parallel)

For Capistrano and it's plugins you are at the mercy of the maintainers
(yours truly) or the plugin authors to have exposed the concurrency
settings.

If there's any more I can help with (long question, short answer!) I'm more
than happy, just drop me a note, you shouldn't have to wait 20h in the
moderation queue whilst I ignore your email next time ;-)

Lee Hambley
http://lee.hambley.name/
+49 (0) 170 298 5667

On 16 March 2016 at 19:24, Eric Rutherford  wrote:

> Versions:
>
>- Ruby 2.0.0
>- Capistrano 3.4.0
>
> I'm trying to work out a solution to limit the number of servers that a
> group of tasks (remove from load balancer, push code, restart service, add
> back to load balancer for example) can run on at a time.  I've been digging
> through past conversations and documentation, but I've failed to find a
> solution with capistrano 3.x so far.  I've found some examples from v2.x,
> but most of the work in that example relied on features that are no longer
> in existence.  I've been working through creating a custom deploy task that
> specifies "in: :sequence" and calls outside tasks, but I'm not sure if I'm
> on the right track and figured that I might ask for some pointers (or a
> push in the right direction) before I head too far in the wrong direction.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Capistrano" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capistrano+unsubscr...@googlegroups.com.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/capistrano/cf49591c-0be0-44d6-b1b3-c44bf2027a91%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capistrano+unsubscr...@googlegroups.com.
To view this discussion on the web, visit 
https://groups.google.com/d/msgid/capistrano/CAN_%2BVLWHyBrT2MzP-e5af79wNarmv6H7MYbLCN4SY4SsHJ1NHQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.