Why do you think that on-the-fly updates won't require implementation and
inputs? It's true that the execution plugin squashes inputs into arguments
and keeps the implementation in arguments.process.script_path, but that's
an implementation detail. Some plugins may care more about what is an input
and what isn't, or do other kinds of mangling to the implementation. In any
case, the user should have to know how to massage these values: they can
set up inputs and possibly configuration (same way as in TOSCA) and then
call Operation.configure to do the right thing.
The implementation and function are not the same, as you can see even in my
trivial example. The implementation is the path to a script. The function
is a path to a Python function. "function" is absolutely not an evaluation
of "implementation": for the execution plugin the function is either
run_script_local or run_script_with_ssh, which is entirely determined by
the topology (relationship to a host node) and has nothing to do with the
implementation. Who knows, some plugins in some cases might totally throw
away the implementation string in some cases if it becomes irrelevant in
some configurations.
I just don't see why we need to introduce restrictions only to avoid saving
a single text column here. As for separating inputs -- seems very obvious
to me.
We will follow up the discussion in person and post our final decision here.
On Tue, May 23, 2017 at 8:57 AM, Ran Ziv <r...@gigaspaces.com> wrote:
> Actually I didn't mean ARIA-180 has already been merged, just that it might
> conflict with it and I think you two should sync about this.
>
> Regarding the new fields - I completely agree with your first paragraph -
> the original values should indeed be kept, and generally speaking the
> service instance models should be decoupled from the template models in
> order to allow on-the-fly updates and so on - but does this really apply
> here? What is the value of storing the original user values on the service
> instance models in this case? On-the-fly updates won't require the original
> value, and it does make sense to me that the only place the original values
> would be stored is on the template. At the end of the day the
> `implementation` and the `function` are one and the same - its simply that
> one of them is already evaluated and the other isn't. Also, Is it not the
> same case as properties that use the `get_input` intrinsic function?
>
> I guess the upside just isn't that clear to me here, and there is the
> downside of these being pretty confusing. The way I see it it's as if
> "implementation" is called "user_implementation" (vs "actual
> implementation") , or rather "template_implementation" - which again hints
> it should sit over at the template models..?
>
>
>
>
>
> On Mon, May 22, 2017 at 7:07 PM, Tal Liron <t...@gigaspaces.com> wrote:
>
> > I see Avia merged ARIA-180, so definitely will convert to one-to-many
> now.
> >
> > The reason the original values are preserved are the same reason all our
> > original values are preserved. The idea is that we don't want to make
> > service instance models dependent on service template models. They are
> > instantiated from them, but these foreign-key fields are all nullable.
> The
> > reason is to allow freedom for orchestration policies with on-the-fly
> > changes the topology. For example, a special kind of scaling policy might
> > involve adding an Nginx load balancer node for HTTP connections. However,
> > what if the user did not have a node template for that kind of the node?
> If
> > we require the creation of an OperationTemplate for every Operation, we
> > make this awkward, because it changes the original template. So, anyway,
> > this has been our principle until now and I suggest we continue this for
> > operations, too.
> >
> > Let's walk through a simple example.
> >
> > 1. A service template with a simple compute node, with a single
> "dazzle"
> > operation on a custom interface, and this "dazzle" has a single
> > implementation string, "dazzle_node.sh", with a single string input:
> > "duration"="forever". To keep things simple, no special configuration
> is
> > added via dependencies.
> > 2. The parser will create an OperationTemplate with these values:
> > 1. implementation = "dazzle_node.sh"
> > 2. inputs = {
> > 1. duration = "forever"}
> > 3. plugin_specification = None
> > 4. configuration = {}
> > 3. During initial instantiation phase, the above values will be
> > copied to Operation model as is. At this early stage, "function" and
> > "arguments" are still both empty.
> > 4. Now during instantiation *configuration* phase, we see that this
> > operation uses the default execution plugin
> (plugin_specification=None).
> > Because this node is a host, it will be configured to be a *remote*
> > operation with very specific arguments. Also, we decided that for all
> > plugins inputs get sent as *extra* arguments. So, here are all final
> > values for Operation:
> > 1. implementation = "dazzle_node.sh"
> > 2. inputs = {
> > 1. duration = "forever"}
> > 3. configuration = {}
> > 4. plugin = None
> > 5. function = "aria.orchestrator.execution_
> > plugin.operations.run_script_with_ssh"
> > 6. arguments = {
> > 1. process = {}
> > 2. script_path = "create_node.sh"
> > 3. use_sudo = False
> > 4. hide_output = []
> > 5. fabric_env = {user: DEFAULT_USER, password: DEFAULT_PASSWORD,
> > key: None, key_filename: None}
> > 6. duration = forever}
> >
> > So, you can see here that if we change "implementation" to be
> > "aria.orchestrator.execution_plugin.operations.run_script_with_ssh"
> (like
> > we did before), the implementation is hidden in one of the arguments
> > (script_path), which would be very tricky for users to discover. That's
> an
> > implementation detail of the execution plugin, not something you want to
> > rely on. Also, if we merge everything into inputs, the original
> distinction
> > between what is an input and what is an implementation detail is lost.
> >
> > The fields in this suggested scheme have distinct and clear meanings:
> >
> > 1. "implementation" and "inputs" are directly from the TOSCA spec.
> They
> > are preserved and never change.
> > 2. "configuration" is an extra optional ARIA extension dict allowing
> > users to configure how the plugin will execute the operation.
> > 3. "function" and "arguments" are internal implementation details of
> the
> > ARIA orchestrator. In a nicely straightforward sense they become a
> > Python
> > function with kwargs.
> >
> > How this clears things up!
> >
> > On Sun, May 21, 2017 at 7:18 AM, Ran Ziv <r...@gigaspaces.com> wrote:
> >
> > > +1 for examples, or perhaps a link to specific code sections.
> > >
> > > From what I did understand though, before we talk about additional
> > changes,
> > > I'm not at all sure I'm in favor of the current ones.
> > >
> > > First, the many-to-many relationship sounds in contrast to what Avia is
> > > currently working on with (changing Parameters relationship type)
> > >
> > > Second, so what you're saying is that now Operation models will have
> both
> > > "implementation" and "function", and both "inputs" and "arguments"?
> This
> > > seems very confusing.
> > > I'm both not convinced the original values are needed at that stage
> (can
> > we
> > > have an actual use case?), but even if they are, do we really need to
> > keep
> > > all four on the same model?
> > > How about having the original values on the OperationTemplate, and the
> > > "coerced values" (for lack of a better term..) on the Operation model?
> > >
> > >
> > >
> > > On Sun, May 21, 2017 at 11:45 AM, Arthur Berezin <
> art...@gigaspaces.com>
> > > wrote:
> > >
> > > > On Sat, May 20, 2017 at 3:47 AM Tal Liron <t...@gigaspaces.com>
> wrote:
> > > >
> > > > > Phew! Everything seems to work now: functions are parsed and
> > evaluated
> > > > > properly. But there are some changes that require explaining.
> > > > >
> > > > > The "configuration" field in OperationTemplate and Operation models
> > is
> > > > now
> > > > > a many-to-many with Parameter. This allows the values to fully
> > support
> > > > > intrinsic functions and types.
> > > > >
> > > > > And I made another change in modeling: Operation now also has
> > > "function"
> > > > > and "arguments" fields, which are what actually gets used by the
> Task
> > > API
> > > > > instead of "implementation" and "inputs". This means that the
> > > > > "implementation" and "inputs" values are preserved as is. During
> the
> > > > > configuration phase the "arguments" are created according to the
> > usual
> > > > > logic: for the execution plugin, the "configuration" is massaged
> > into a
> > > > few
> > > > > new "arguments", and otherwise other "inputs" are also merged into
> > > > > "arguments".
> > > > >
> > > > > Why make this change? Because I really think we should not mangle
> the
> > > > > "inputs": they are logically different from the arguments that get
> > sent
> > > > to
> > > > > the @operation function, even if in many cases we might treat them
> > > > > identically. Also, we cannot lose the "implementation" string:
> > plugins
> > > > will
> > > > > need to know what users put in there. The change is not big in
> terms
> > of
> > > > > extra code, but I think it helps make the code easier to
> understand:
> > > you
> > > > > can see exactly what is being merged into the final "arguments" for
> > the
> > > > > task. The configure phase is now constructive rather than
> > destructive:
> > > > > existing fields are not changed.
> > > > >
> > > >
> > > >
> > > > Tal, could you provide several example models to showcase these
> > changes?
> > > >
> > > >
> > > >
> > > > >
> > > > > Before moving forward with a PR I want to see what other committers
> > > think
> > > > > about also renaming "implementation" and "inputs" in the Task API
> to
> > > > > "function" and "arguments". For now I'm leaving them as is, but I
> > think
> > > > it
> > > > > would be even cleaner if they followed the new naming convention in
> > > > > modeling. I continue being unhappy about how the Task API treats
> both
> > > > > arguments and inputs as the same thing and squashing them all
> > > together...
> > > > > :( However, for now I would be happy if we just called them
> > > "arguments",
> > > > > because that's what they really are for us.
> > > > >
> > > > > --
> > > > > Tal Liron
> > > > > Solution Architect
> > > > > t...@gigaspaces.com | +13123758299 <(312)%20375-8299>
> > > > > Cloudify | http://getcloudify.org
> > > > > <
> > > > > http://getcloudify.org?utm_source=signaturesatori&utm_
> > > > medium=email&utm_campaign=general_signature
> > > > > >
> > > > >
> > > > > <https://twitter.com/CloudifySource>
> > > > > <https://www.linkedin.com/groups/8467478>
> > > > > <https://github.com/cloudify-cosmo> <
> https://github.com/cloudify-
> > > cosmo
> > > > >
> > > > > [image: Azure Webinar]
> > > > > <
> > > > > http://getcloudify.org/webinars/Azure-plugin-for-
> > > > cloudify-webinar.html?utm_source=signaturesatori&utm_
> > > > medium=email&utm_campaign=general_signature
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Tal Liron
> > Senior Engineer
> > t...@gigaspaces.com | +1 (773) 828-9339
> > Cloudify | http://getcloudify.org
> > <http://getcloudify.org?utm_source=signaturesatori&utm_
> > medium=email&utm_campaign=general_signature>
> >
> > <https://twitter.com/CloudifySource>
> > <https://www.linkedin.com/groups/8467478>
> > <https://github.com/cloudify-cosmo> <https://github.com/cloudify-cosmo
> >
> > [image: Azure Webinar]
> > <http://getcloudify.org/webinars/Azure-plugin-for-
> > cloudify-webinar.html?utm_source=signaturesatori&utm_
> > medium=email&utm_campaign=general_signature>
> >
>
--
Tal Liron
Senior Engineer
t...@gigaspaces.com | +1 (773) 828-9339
Cloudify | http://getcloudify.org
<http://getcloudify.org?utm_source=signaturesatori&utm_medium=email&utm_campaign=general_signature>
<https://twitter.com/CloudifySource>
<https://www.linkedin.com/groups/8467478>
<https://github.com/cloudify-cosmo> <https://github.com/cloudify-cosmo>
[image: Azure Webinar]
<http://getcloudify.org/webinars/Azure-plugin-for-cloudify-webinar.html?utm_source=signaturesatori&utm_medium=email&utm_campaign=general_signature>
