Notice the section in http://docs.fabfile.org/en/1.4.1/api/core/tasks.html#fabric.tasks.execute where it mentions that "Any other arguments or keyword arguments will be passed verbatim into task when it is called, so execute(mytask, 'arg1', kwarg1='value') will (once per host) invoke mytask('arg1', kwarg1='value')."
It is refering, albeit not clearly, to the fact that any keyword that is given will be used as if it were a member of the env dict. So you can use any of these env var names to specify things, eg: parallel=True; http://docs.fabfile.org/en/1.4.1/usage/env.html#full-list-of-env-vars Also I fumbled the name of the decorator. It's serial: http://docs.fabfile.org/en/1.4.1/api/core/decorators.html#fabric.decorators.serial You can also specify custom args to fab tasks: http://docs.fabfile.org/en/1.4.1/usage/fab.html#per-task-arguments And with that you could make the deploy task take a role name, or list of hosts to then use as the lists, local and remote, to use in your script. -goose On Tue, May 1, 2012 at 12:01 PM, anatoly techtonik <[email protected]> wrote: > Hi Morgan, > > I've tried your example, which is: > > local = ['a','b'] > remote = ['c','d'] > > def test:pass > def update:pass > > @task > @sequential > def deploy(): > execute(test, hosts=local+remote) > execute(update, hosts=local+remote) > > At first it didn't run at all with Fabric 1.4.1, unable to call > @sequential decorator: > > Traceback (most recent call last): > ... > File "/checkout83/fabfile.py", line 20, in <module> > @sequential > NameError: name 'sequential' is not defined > > I commented the @sequential, after which `fab deploy` run sequentially: > local test > remote test > local update > remote update > > That's much better, but it still impossible to specify hosts from > command line. `fab -H local deploy` still runs additional command on > the remote, and `fab -H local,remote deploy` run everything twice on > both hosts. > > I couldn't find parallel among params for execute - > http://docs.fabfile.org/en/1.4.1/api/core/tasks.html#fabric.tasks.execute > so I've just decorated functions: > > local = ['a','b'] > remote = ['c','d'] > > @task > @parallel > def test:pass > @task > @parallel > def update:pass > > @task > def deploy(): > execute(test, hosts=local+remote) > execute(update, hosts=local+remote) > > I must admit that although not ideally, but this works. I just need to > make sure that deploy is always called with empty host list and I need > to find a way to specify hosts from command line for the subtasks. > -- > anatoly t. > > > On Fri, Apr 27, 2012 at 11:48 PM, Morgan Goose <[email protected]> wrote: >> The example I gave you and how to run it did all of those things, sans >> the parallel. Execute has parallel as a param. Use that in your master >> task's executes, and you'll have everything. If that still isn't what >> you're looking for, we're going to need more information. Eg, like you >> did before where you said how you're seeing it run, and how you'd >> instead like it to run, complete with how you ran it with fab, and how >> the fabfile looks. >> >> -goose >> >> On Thu, Apr 26, 2012 at 10:32 PM, anatoly techtonik <[email protected]> >> wrote: >>> Thanks for the explanation. I read the tutorial. Still I see I can't >>> construct the system that will satisfy the following usability >>> requirements: >>> >>> 1. Define (or override) hosts from command line >>> 2. Execute each scheduled task in parallel >>> 3. Wait until previous task completes successfully on all servers >>> before moving to the next one >>> 4. Ability to run each task separately or use master task as a helper >>> command >>> >>> Command line hosts (1) is needed to test new nodes and control >>> deployment from 3rd party application. >>> Parallel execution (2) and (3) is critical to minimize servers downtime. >>> Master task (4) is also highly desired, because full deployment process >>> contains a lot of steps. >>> >>> It seems that this could be real with the @mastertask decorator that >>> would make task run without servers at all. With it the following >>> could work as expected. >>> >>> @task >>> @parallel >>> def test:pass >>> >>> @task >>> @parallel >>> def update:pass >>> >>> @mastertask >>> def deploy(): >>> execute(test) >>> execute(update) >>> >>> -- >>> anatoly t. >>> >>> >>> On Fri, Apr 27, 2012 at 1:28 AM, Morgan Goose <[email protected]> >>> wrote: >>>> I just read that last line, where you said you wanted to run them with >>>> hosts defined at runtime. You can either use functions to generate the >>>> hosts lists or with said example i gave change it up to get rid of the >>>> deploy task, and then run the fabfile like: >>>> $ fab -H local,remote test update >>>> >>>> As that will be sequential and honor the host list as well. It will >>>> also run the test on all hosts before moving to the next task listed, >>>> update. >>>> >>>> -goose >>>> >>>> On Thu, Apr 26, 2012 at 3:25 PM, Morgan Goose <[email protected]> >>>> wrote: >>>>> First have you read the tutorial? This is the section that will get >>>>> you in on host list defining, and point you the the more detailed >>>>> information that I link to below it: >>>>> http://docs.fabfile.org/en/1.4.1/tutorial.html#defining-connections-beforehand >>>>> http://docs.fabfile.org/en/1.4.1/usage/execution.html#host-lists >>>>> >>>>> As to what you want to do. Each task will loop over it's list of hosts >>>>> to run on. So top down it's TaskA over all HostListA's hosts. Then if >>>>> no errors move to the other task with the same or unique host list >>>>> (defined either globally or per task). >>>>> >>>>> From what you're wanting to do you will construct the fabfile like this: >>>>> >>>>> local = ['a','b'] >>>>> remote = ['c','d'] >>>>> >>>>> def test:pass >>>>> def update:pass >>>>> >>>>> @task >>>>> @sequential >>>>> def deploy(): >>>>> execute(test, hosts=local+remote) >>>>> execute(update, hosts=local+remote) >>>>> >>>>> >>>>> Then all hosts will have the test task run on them before moving to >>>>> looping over the two hosts lists and running the update. You then call >>>>> it simply with: >>>>> >>>>> $ fab deploy >>>>> >>>>> And you never run the test and update tasks themselves. Nor do you >>>>> define anything to be parallel. As parallel runs on the task would >>>>> test on both hosts in parallel, and then deploy to both hosts in >>>>> parallel. This was, running sequential and explicitly defined to do >>>>> so, will fail fast on the test hosts if one dies. Because the host >>>>> list's order is honored. >>>>> >>>>> -goose >>>>> >>>>> On Wed, Apr 25, 2012 at 1:13 PM, anatoly techtonik <[email protected]> >>>>> wrote: >>>>>> On Sat, Apr 21, 2012 at 7:51 PM, Jeff Forcier <[email protected]> >>>>>> wrote: >>>>>>> On Fri, Apr 20, 2012 at 5:31 AM, anatoly techtonik >>>>>>> <[email protected]> wrote: >>>>>>> >>>>>>>> Is it possible in fabric to wait until a subtask completes on all >>>>>>>> servers successfully before moving to next step? >>>>>>> >>>>>>> You need to use execute() to treat subroutines as if they were full >>>>>>> fledged tasks. execute() is the machinery that says "take this >>>>>>> function and run it once per host in this list of hosts." Right now >>>>>>> that machinery is just applying implicitly to your deploy() task and >>>>>>> you're probably using env.hosts to set your host list. >>>>>>> >>>>>>> Remove env.hosts and maybe make that host list a role in env.roledefs. >>>>>>> Then you can do this: >>>>>>> >>>>>>> env.roledefs = {'myrole': ['a', 'b', 'c']} >>>>>>> >>>>>>> def test(): ... >>>>>>> def update(): ... >>>>>>> >>>>>>> def deploy(): >>>>>>> execute(test, role='myrole') >>>>>>> execute(update, role='myrole') >>>>>>> >>>>>>> That should have the effect you want: "fab deploy" => first test() >>>>>>> runs once per host, and when it's all done, update() will run once per >>>>>>> host. deploy() itself will end up running only one time total -- it's >>>>>>> just a "meta" task now. >>>>>> >>>>>> It took time to realize the that execute() iterates over the global >>>>>> list of hosts. I expected that the following two to be equivalent, but >>>>>> they were not: >>>>>> >>>>>> 1. fab -H local,remote test update >>>>>> 2. fab -H local,remote deploy >>>>>> >>>>>> I used the script without roles: >>>>>> >>>>>> def test(): ... >>>>>> def update(): ... >>>>>> def deploy(): >>>>>> execute(test) >>>>>> execute(update) >>>>>> >>>>>> 1st execution variant is fully synchronous (i.e. next task doesn't >>>>>> start until previous finishes) and gave the sequence: >>>>>> >>>>>> local test >>>>>> local update >>>>>> remote test >>>>>> remove update >>>>>> >>>>>> but the 2nd variant with subtasks was confusing (I indented to see >>>>>> what's going on): >>>>>> >>>>>> local deploy >>>>>> local test >>>>>> remote test >>>>>> local update >>>>>> remote update >>>>>> remote deploy >>>>>> local test >>>>>> remote test >>>>>> local update >>>>>> remote update >>>>>> >>>>>> I found fabric pretty counter-intuitive in this case. I tried to fix >>>>>> that without roles by explicitly passing current host: >>>>>> >>>>>> def test(): ... >>>>>> def update(): ... >>>>>> def deploy(): >>>>>> execute(test, host=env.host_string) >>>>>> execute(update, host=env.host_string) >>>>>> >>>>>> This gives: >>>>>> local deploy >>>>>> local test >>>>>> local update >>>>>> remote deploy >>>>>> remote test >>>>>> remove update >>>>>> >>>>>> Still not the desired behavior. The desired is: >>>>>> deploy >>>>>> local test >>>>>> remote test >>>>>> wait >>>>>> local update >>>>>> remote update >>>>>> >>>>>> I've tried using @parallel decorator for deploy task and it seemed to >>>>>> work fine at first. >>>>>> local deploy >>>>>> remote deploy >>>>>> remote test >>>>>> local test >>>>>> local update >>>>>> remote update >>>>>> >>>>>> But the test step didn't not synchronize - local update executed while >>>>>> remote test was still running. It looks like the roles is the only >>>>>> chance, but I'd like to avoid hardcoding server names at all costs. Is >>>>>> it possible? >>>>>> >>>>>> -- >>>>>> anatoly t. >>>>>> >>>>>> _______________________________________________ >>>>>> Fab-user mailing list >>>>>> [email protected] >>>>>> https://lists.nongnu.org/mailman/listinfo/fab-user _______________________________________________ Fab-user mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/fab-user
