Hi all,
Great discussion - really like the direction this is going of using
tasks in yaml blueprints.
However, it feels like we've launched into discussing a complex use-case
(concurrency control) without having first discussed what yaml
blueprints would look like for simpler tasks (*). I suggest we discuss
that in another proposal thread, and get agreement of what the YAML
should look like. Let's focus on the yaml blueprint author - i.e. reach
agreement on some yaml examples, without worrying too much about how
it's implemented under the covers.
I'd like the examples to show an entire blueprint - including defining
effectors, sensors, feeds, etc.
---
Svet, for the "callback behaviour for effectors" I think that would be a
powerful/advanced feature. If multiple callbacks were registered (e.g.
in the super-type and in the sub-type, when extending existing
blueprints), the order the callbacks are called could get tricky for
blueprint authors. Maybe we live with that. Maybe the task-based
approach will give a more elegant way to control it, making the
callbacks redundant.
If it helps to solve your immediate problem cleanly, I'm fine with us
adding support for it. But I worry that we'd want to deprecate it
shortly after task-based yaml is supported.
Aled
(*) Sorry if I've forgotten about some previous discussion on the
mailing list - if I have, then please point us at it again!
On 11/01/2017 17:45, Svetoslav Neykov wrote:
Sam I think that's a great approach to composing tasks and mixing them with
semaphores.
sem --id brooklyn --fg "$REMOVE_OLD_CMD ; $ADD_NEW_CMD"
Alex, I like that, gives you much cleaner semantics than flock.
effectors:
start:
my-pre-launch:
task: { acquire-semaphore: ... }
run-before: launch
Geoff, I have some reservations re init.d style naming as well. Mostly due to
being too fragile when having multiple layers of blueprints. I'm going to
suggest one more alternative, somehow similar to before/after. Instead of
declaring when to run a task (before/after another task), just override the
parent task, do the work you need to and reference the parent task explicitly.
effectors:
start:
launch: [my-custom-launch, $super.launch]
my-custom-launch:
- ssh: ....
Anyway this is getting sidetracked. I agree that the task based approach is a
nice place to be longer term. Might go with extending the latches shorter term
- need to think some more about it.
What do people think about introducing the callback behaviour for effectors though? I see
it as an orthogonal concept to what's possible with composing tasks. It's a middle ground
for implementing the semaphores where the latches are too limited and "semaphores as
tasks" are still not here. Will make life easier for some more complex scenarios
where one needs to sync with/influence other entities (examples in previous emails).
Svet.
On 11.01.2017 г., at 18:50, Alex Heneveld <[email protected]>
wrote:
Svet-
On 11/01/2017 15:55, Svetoslav Neykov wrote:
PS - as a near-term option if needed we could extend SoftwareProcess LATCH to do
something special if the config/sensor it is given is a "Semaphore" type
What do you think the behaviour should be here - releasing the semaphore after
the corresponding step completes or at the end of the wrapping effector? I
think this is defined by the blueprint authors. And making this configurable
adds even more complexity. Instead could invest in developing the above
functionality.
I'd probably release the latch sempahore right after the phase but I agree it's
arbitrary and don't like that. Only suggested it if we need something very
quick as it's isolated and probably fairly easy in the code.
Note code exists and is run implicitly I think during the Install step
(otherwise things like yum complain), along with I think the class is called
SemaphoreWithOwner.
I'd LOVE to invest in the task-based functionality. Code-wise it's not that
far away with the type registry and YOML, but it needs a couple more people to
become familiar with it!
Lastly I meant to say -- someone mentioned flock but the bash *sem* command
(aka parallel) is awesome though not as well known as it should be. This is an
example used in zookeeper:
sudo yum install parallel
REMOVE_OLD_CMD="sed -i /server.*=/d zoo.cfg"
ADD_NEW_CMD="cat >> zoo.cfg << EOF
server.0=NEW
server.1=NEW
EOF
"
sem --id brooklyn --fg "$REMOVE_OLD_CMD ; $ADD_NEW_CMD"
It runs the commands in the quotes on the last line while acquiring the semaphore with
the given name "brooklyn" on the machine. You can configure semaphore counts
and timeouts too.
Best
Alex
