+1 to Alex's core suggestion. It would be a powerful extension to
Brooklyn. We should not force blueprint authors into arbitrary
structures that were only a good fit for someone else's scenario in the
past.
To lob another suggestion into the mix, I imagine writing:
effectors:
# default named phases, not needed normally.
start: [provision, install, customise, launch]
# launch phase overridden to wrap latter half with mutex handling
launch:
- ssh-task-1
- mutexed
subtasks:
- ssh-task-2
- wait-for-running
This implies that each of the phases of start is another effector and
that they are configurable objects. The `mutexed` task cleans up
properly when its subtasks fail.
Sam
On 11/01/2017 15:55, Svetoslav Neykov wrote:
Alex,
I like the task based approach and agree it's something we need to push for.
I'm not convinced that locking-like behaviour should be represented as tasks
though. It brings procedural style flavour to concurrency - for example in your
original example there's a separate tasks for acquiring and releasing the lock.
They could easily get misplaced, overwritten, forgotten, etc. Not clear how
releasing it works in case of errors.
What I'd prefer is more declarative approach where the tasks are grouped and
the locking requirements are applied on the group. At worst referencing a start
task and optionally an end task (with the default of the parent task existing).
Plugging into the lifecycle of entities has other use-cases. No matter how an entity is defined - whether
using the current monolithic START effector or one composited of smaller tasks - there's no way to be
notified when its tasks get executed. Some concrete examples - the "SystemServiceEnricher" which
looks for the "launch" task and can be applied on any "SoftwareProcessEnity"; An entity
which needs to do some cleanup based on the shutdown of another entity (DNS blueprints); latching at
arbitrary points during entity lifecycle; etc.
One of the alternatives I mention in the original email (Effector execution
notifications) is a step in the "task based" approach direction. It's still in
Java, but we are splitting the monolith effectors into smaller building blocks, which
could in future get reused in YAML.
So to summarise - +1 for tasks as building blocks, but still need visibility
into the executing blocks.
PS - as a near-term option if needed we could extend SoftwareProcess LATCH to do
something special if the config/sensor it is given is a "Semaphore" type
What do you think the behaviour should be here - releasing the semaphore after
the corresponding step completes or at the end of the wrapping effector? I
think this is defined by the blueprint authors. And making this configurable
adds even more complexity. Instead could invest in developing the above
functionality.
Svet.
On 11.01.2017 г., at 16:25, Alex Heneveld <[email protected]>
wrote:
svet, all,
lots of good points.
the idea of "lifecycle phases" in our software process entities has grown to be something
of a monster, in my opinion. they started as a small set of conventions but they've grown to the
point where it's the primary hook for yaml and people are wanting "pre.pre.install".
it's (a) misguided since for any value of N, N phases is too few and (b) opaque
coming from a java superclass.
a lot of things rely on the current SoftwareProcess so not saying we kill it, but for the community it would
be much healthier (in terms of consumers) to allow multiple strategies and especially focus on *the
reusability of tasks in YAML* -- eg "provision" or "install template files" or
"create run dir" -- so people can write rich blueprints that don't require magic lifecycle phases
from superclasses.
the "init.d" numbered approach svet cites is one way these are wired together,
with extensions (child types) able to insert new numbered steps or override with the
number and label. but that would be one strategy, there might be simpler ones that are
just a list, or other sequencing strategies like precondition/action/postcondition or
runs-before/runs-after (where to extend something, you can say `my-pre-customize: { do:
something, run-before: launch, run-after: customize }`).
we're not sure exactly how that would look but wrt mutex logic the idea that
adjuncts plug into the entity lifecycles feels wrong, like it's pushing for
more standardisation in lifecycle phases where things can plug in. whereas
with a task approach we can have effector definitions being explicit about
synchronization, which i think is better. and if they want to make that
configurable/pluggable they can do this and be explicit about how that is done
(for instance it might take a config param, even a config param (or child or
relation) which is an entity and call an effector on that if set).
concretely in the examples i'm saying instead of ENRICHER APPROACH
memberSpec:
$brooklyn:entitySpec:
type: cluster-member
brooklyn.enrichers:
- type: org.apache.brooklyn.enricher.stock.AquirePermissionToProceed
brooklyn.config:
stage: post.provisioning
# lifecycle stage "post.provisioning" is part of cluster-member
and
# the enricher above understands those stages
we concentrate on a way to define tasks and extend them, so that we could
instead have a TASK APPROACH:
memberSpec:
$brooklyn:entitySpec:
type: cluster-member
effectors:
start:
035-pre-launch-get-semaphore: { acquire-semaphore: ... }
# assume 040-launch is defined in the parent "start" yaml defn
# using a new hypothetical "initd" yaml-friendly task factory,
# acquire-semaphore is a straightforward task;
# scope can come from parent/ancestor task,
# also wanted to ensure mutex is not kept on errors
or
memberSpec:
$brooklyn:entitySpec:
type: cluster-member
effectors:
start:
my-pre-launch:
task: { acquire-semaphore: ... }
run-before: launch
run-after: customize
# launch and customize defined in the parent "start" yaml
defn,
# using a new hypothetical "ordered-labels" yaml-friendly
task factory;
# acquire-semaphore and scope are as in the "initd" example
both approaches need a little bit of time to get your head around the new
concepts but the latter approach is much more powerful and general. there's a
lot TBD but the point i'm making is that *if we make tasks easier to work with
in yaml, it becomes more natural to express concurrency control as tasks*.
PS - as a near-term option if needed we could extend SoftwareProcess LATCH to do
something special if the config/sensor it is given is a "Semaphore" type
best
alex
