Re: Service Composition / Substitution Mapping

[Tal Liron]

"directives" is just a list of strings. Moreover, it seems like a
placeholder for something in the future. Also note that "directives" exist
only for node templates, not node type, so this means that in order to use
substitution we would have to have a node template marked as being
abstract. If we want to use a directive with some sophistication, we would
have to invent some weird string-based language for this (like we did for
operation configuration) in order to configure exactly how substitution
would happen.

I really think that a policy is the correct way to handle this for now, and
possibly for the future. It is the most "TOSCA-in-spirit" solution.

On Wed, Sep 6, 2017 at 3:45 AM, Avia Efrat  wrote:

> On Tue, Sep 5, 2017 at 6:09 PM, Tal Liron  wrote:
>
> >
> > Just like you can specify a specific node template for a requirement, I
> > believe it can be useful and often necessary to specify a substituted
> > service template.
> >
>
> ​Even​ if it is needed, the question is how to do that. Using a policy
> seems too much like a catch-all solution. In the spec, there is the notion
> of the "substitutable" directive [2.10.2]. At first I thought it was meant
> as a general concept, but then I noticed that:
>
> 1. node templates have a `directives` field, that is described as "an
> optional list of directive values to provide processing instructions to
> orchestrators and tooling" [3.7.3.1].
> 2. section [2.10.2] does not mention "substitutable directive", but rather
> "substitutable" directive. It seems that their intention is the adding the
> "substitutable" string to the `directives` field of the node template is
> the mean by TOSCA is designating a node to be substituted.
> 3. section [2.9] extensively deals with abstract nodes. It mentions another
> use of the `directives` field, this time the "selectable" directive (also
> in [3.7.3.1], [3.7.3.3]).
>
> Although section [3.3.3] states that there are no defined directives in
> TOSCA 1.0, the de facto usages of the directive field indicates at least
> that there is intent of using this field, and maybe define it more
> explicitly in future versions.
>
> Bottom line, currently I see the `directives` field as the the indicator of
> substitutable nodes (by specifying the "substitutable" directive). It is
> needed to enquire OASIS regarding their future plans of TOSCA.
>

Re: Service Composition / Substitution Mapping

[Avia Efrat]

On Tue, Sep 5, 2017 at 6:09 PM, Tal Liron  wrote:

>
> Just like you can specify a specific node template for a requirement, I
> believe it can be useful and often necessary to specify a substituted
> service template.
>

​Even​ if it is needed, the question is how to do that. Using a policy
seems too much like a catch-all solution. In the spec, there is the notion
of the "substitutable" directive [2.10.2]. At first I thought it was meant
as a general concept, but then I noticed that:

1. node templates have a `directives` field, that is described as "an
optional list of directive values to provide processing instructions to
orchestrators and tooling" [3.7.3.1].
2. section [2.10.2] does not mention "substitutable directive", but rather
"substitutable" directive. It seems that their intention is the adding the
"substitutable" string to the `directives` field of the node template is
the mean by TOSCA is designating a node to be substituted.
3. section [2.9] extensively deals with abstract nodes. It mentions another
use of the `directives` field, this time the "selectable" directive (also
in [3.7.3.1], [3.7.3.3]).

Although section [3.3.3] states that there are no defined directives in
TOSCA 1.0, the de facto usages of the directive field indicates at least
that there is intent of using this field, and maybe define it more
explicitly in future versions.

Bottom line, currently I see the `directives` field as the the indicator of
substitutable nodes (by specifying the "substitutable" directive). It is
needed to enquire OASIS regarding their future plans of TOSCA.

RE: Service Composition / Substitution Mapping

[D Jayachandran]

Thanks Tal. I agree this needs to taken up with OASIS.

Having said that ,there is statement in TOSCA Spec 2.9 

" In TOSCA templates, nodes are either:

· Concrete: meaning that they have a deployment and/or one or more 
implementation artifacts that are declared on the “create” operation of the 
node’s Standard lifecycle interface, or they are

· Abstract: where the template describes the node type along with its 
required capabilities and properties that must be satisfied."

I thought if the above statement would anyway help us in identifying an 
abstract node.


Regards,
DJ

-Original Message-
From: Tal Liron [mailto:t...@cloudify.co] 
Sent: Tuesday, September 05, 2017 8:30 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping

On Tue, Sep 5, 2017 at 5:33 AM, D Jayachandran <d.jayachand...@ericsson.com>
wrote:

>  Hence I wanted to know if there is any possibility of identifying an 
> abstract type (during parsing )so that we need not import or define 
> the custom node type in my service template.
>

In my opinion, the import of the abstract type is necessary. TOSCA offers no 
alternative, and I don't see how ARIA would be able to guess that this type is 
abstract nor have any notion where to find the type. I think that such 
usability problems we find in the mechanism need to be addressed at OASIS. 
Perhaps we can make TOSCA 1.2 easier to use with substitution mapping. For now, 
though, I think ARIA should adhere to the TOSCA 1.0 standard as closely as 
possible.

Re: Service Composition / Substitution Mapping

[Tal Liron]

On Sun, Sep 3, 2017 at 4:06 AM, Ran Ziv  wrote:

> I too agree with the notion that picking a substituting service template
> explicitly is against the spirit of this feature, for the reasons Avia has
> explained.
> If a specific template is needed, why not import it directly, or require a
> more specific capability offered by the substituting service template?
>

Just like you can specify a specific node template for a requirement, I
believe it can be useful and often necessary to specify a substituted
service template.

Let me try an example:

topology_template:
  node_templates:
my_app:
  type: WebApp
  requirements:
host:
  capability: tosca.capabilities.Container
  node: my_host # explicit node template
db:
  capability: MongoDb
my_host:
  type: VirtualMachine

  policies:
substitution:
  type: aria.Substitution
  targets: [ my_app ]
  properties:
requirements: # map of strings to requirement policy infos
  db:
service_template: /opt/templates/MongoDbCluster.csar # explicit
service template

Re: Service Composition / Substitution Mapping

[Tal Liron]

On Tue, Sep 5, 2017 at 5:33 AM, D Jayachandran 
wrote:

>  Hence I wanted to know if there is any possibility of identifying an
> abstract type (during parsing )so that we need not import or define the
> custom node type in my service template.
>

In my opinion, the import of the abstract type is necessary. TOSCA offers
no alternative, and I don't see how ARIA would be able to guess that this
type is abstract nor have any notion where to find the type. I think that
such usability problems we find in the mechanism need to be addressed at
OASIS. Perhaps we can make TOSCA 1.2 easier to use with substitution
mapping. For now, though, I think ARIA should adhere to the TOSCA 1.0
standard as closely as possible.

RE: Service Composition / Substitution Mapping

[D Jayachandran]

Hi All,

Thanks for the comments on handling substitution when multiple service 
templates matches for the abstract node type.

My 2nd point was on importing the non-normative node types in the top level 
service template. Let me explain this with an example.

1)  Normative type (tosca.nodes.Database) as abstract node
  db:
  # This node is abstract (no Deploment or Implemenation artifacts on 
create)
  # and can be substituted with a topology provided by another template
  # that exports a Database type’s capabilities.
  type: tosca.nodes.Database
  properties:
user: my_db_user
password: secret
name: my_db_name

With this example, the abstract node type is a normative type from the simple 
YAML 1.0 profile. Hence with our proposed approach we will go through the 
available service templates and find the required substitution.

2) Non-normative type (myapp.nodes.Database)
If I replace the above example with the non-normative type " 
myapp.nodes.Database" , we need to import this type in our service template to 
have the parsing complete. 

import types/my_custom.yaml
 
node_templates: 
  db:
  # This node is abstract (no Deploment or Implemenation artifacts on 
create)
  # and can be substituted with a topology provided by another template
  # that exports a Database type’s capabilities.
  type: myapp.nodes.Database
  properties:
user: my_db_user
password: secret
name: my_db_name

Here "db" is an abstract node of type  "myapp.nodes.Database" which needs to be 
substituted from other service templates. Though it will be substituted we need 
to have this type "myapp.nodes.Database" defined in my top level service 
template or imported for the parsing to be sucessfull.  Hence I wanted to know 
if there is any possibility of identifying an abstract type (during parsing )so 
that we need not import or define the custom node type in my service template. 

Regards,
DJ
-Original Message-
From: Ran Ziv [mailto:r...@cloudify.co] 
Sent: Sunday, September 03, 2017 2:36 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping

I'm not sure I've followed why a policy would be necessary.

I too agree with the notion that picking a substituting service template 
explicitly is against the spirit of this feature, for the reasons Avia has 
explained.
If a specific template is needed, why not import it directly, or require a more 
specific capability offered by the substituting service template?


On Thu, Aug 31, 2017 at 7:09 PM, Tal Liron <t...@cloudify.co> wrote:

> Regarding #1, I agree with Avia here. Substitution mapping is intended 
> as an extension of requirements-matching-capabilities. As you know, a 
> requirement can be as minimal as just specifying a capability type, 
> but could optionally additionally specify a node and even include a 
> node filter with more constraints. Unfortunately the substitution 
> mapping grammar is still not very sophisticated in TOSCA 1.0, and 
> while you can require a node, you can't specifically require it to be 
> a substituted service template. That's why I'm thinking we may need to 
> introduce a policy to fill in this gap for now. But indeed, if there 
> is no such policy, available service templates could be tried 
> arbitrarily one at a time until one satisfies the requirement (as Avia 
> points out, this would require the composite service to be *entirely* 
> successfully satisfied).
>
> On Thu, Aug 31, 2017 at 10:23 AM, Avia Efrat <a...@cloudify.co> wrote:
>
> > 1) Substitutable nodes (abstract nodes) are defined by their type. 
> > And a node type should represent specific node/subsystem characteristics.
> Having
> > two node types ( with the same name) that represent different 
> > logical entities seems odd, and indicates a possible bug in the 
> > design. That is another reason why using so-called namespaces while 
> > creating type hierarchies is a good practice.
> >
> > You could argue that if you have multiple sources for templates (e.g.
> your
> > own storage and a external catalog), there could be unexpected name 
> > collisions of node types names. Having said that, we need to 
> > remember the entire process of the substitution mapping.
> >
> > First, as you said, we are trying to locate a service template that 
> > has a substitution mapping definition of the same type of the 
> > abstract node in the top level template. But choosing the service 
> > template is only the
> first
> > step. The second step is instantiating the top level template along 
> > with the substitution template. During the instantiation, 
> > requirements and capabilities are matched. If not all requirements 

Re: Service Composition / Substitution Mapping

[Ran Ziv]

to implicitly import that custom node type in our top level
> service
> > > template so that the parser recognizes this custom type.
> > > Assuming the abstract node type would be substituted from a
> > stored
> > > substituting service template, we need to at least import the custom
> node
> > > types and have it part of the same CSAR package.
> > > Would this be a challenge for the top-level service template
> > > author in including and importing the custom node types as abstraction
> ?
> > or
> > > Is this how we are looking at custom node types ?
> > > Is it possible to identify an abstract node during parsing ,
> such
> > > as if it does not contain any implementation ( The SPEC does not say
> > > anything on this )?
> > >
> > > Regards,
> > > DJ
> > > -Original Message-
> > > From: Ran Ziv [mailto:r...@cloudify.co]
> > > Sent: Wednesday, August 16, 2017 6:19 PM
> > > To: dev@ariatosca.incubator.apache.org
> > > Subject: Re: Service Composition / Substitution Mapping
> > >
> > > I agree, especially when the benefit of being able to use an existing
> > > service - yet only one which hasn't been deployed via a workflow -
> > doesn't
> > > seem all that interesting IMO.
> > >
> > > Another concern I could add to the ones you've mentioned is the
> service's
> > > inputs - the substituting template's inputs should be received via the
> > > properties of the abstract node in the top level service template. If
> the
> > > service already exists, these inputs would not be passed as expected.
> > >
> > > Ran
> > >
> > > On Wed, Aug 16, 2017 at 3:25 PM, D Jayachandran <
> > > d.jayachand...@ericsson.com
> > > > wrote:
> > >
> > > > Hi Ran,
> > > >
> > > > When Tal mentioned about "substituting service", I thought it was
> > > > about the services which dint have any associated
> executions/workflows
> > > triggered.
> > > > Am also in favor of  "substituting service templates" rather than
> > > > "substituting service".
> > > > With "substituting service" approach (when the service is not
> > > > instantiated), I see some open points
> > > > - In a multi-user scenario, what will happen when a service
> is
> > > > composed using the substituting service and at the sametime a
> > > > workflow is triggered for the substituting service. ?
> > > > - Is it okay to delete(dissolve) the substituting service
> > > > after it is used to create the composed service. ?
> > > >
> > > > Starting with it might be a good idea to only have "substituting
> > > > service templates" approach.
> > > >
> > > > Regards,
> > > > DJ
> > > > -Original Message-
> > > > From: Ran Ziv [mailto:r...@cloudify.co]
> > > > Sent: Wednesday, August 16, 2017 4:29 PM
> > > > To: dev@ariatosca.incubator.apache.org
> > > > Subject: Re: Service Composition / Substitution Mapping
> > > >
> > > > I'd say right now we're looking at "static service composition" which
> > > > is only about "substituting service templates", not "substituting
> > > > service". If a service is already running, it will not be used.
> > > >
> > > > I think what Tal meant was that each service template - whether the
> > > > top level one or one of the substituting templates - needs to resolve
> > > > its inner reqs internally first, and then resolve substitution
> > > > reqs across service templates.
> > > >
> > > >
> > > > On Wed, Aug 16, 2017 at 12:00 PM, D Jayachandran <
> > > > d.jayachand...@ericsson.com> wrote:
> > > >
> > > > > Hi Tal,
> > > > >
> > > > > Thanks for organizing the points.
> > > > > So if I understand correctly we are looking only at "Static service
> > > > > composition" which includes "substituting service template" and
> > > > > "substituting service".
> > > > >
> > > > > As you said with "substituting service template" approach ,we will
> > > > > have all the nodes aggregated from other service templates and a
> > > > > single workflow would 

Re: Service Composition / Substitution Mapping

[Tal Liron]

Regarding #1, I agree with Avia here. Substitution mapping is intended as
an extension of requirements-matching-capabilities. As you know, a
requirement can be as minimal as just specifying a capability type, but
could optionally additionally specify a node and even include a node filter
with more constraints. Unfortunately the substitution mapping grammar is
still not very sophisticated in TOSCA 1.0, and while you can require a
node, you can't specifically require it to be a substituted service
template. That's why I'm thinking we may need to introduce a policy to fill
in this gap for now. But indeed, if there is no such policy, available
service templates could be tried arbitrarily one at a time until one
satisfies the requirement (as Avia points out, this would require the
composite service to be *entirely* successfully satisfied).

On Thu, Aug 31, 2017 at 10:23 AM, Avia Efrat <a...@cloudify.co> wrote:

> 1) Substitutable nodes (abstract nodes) are defined by their type. And a
> node type should represent specific node/subsystem characteristics. Having
> two node types ( with the same name) that represent different logical
> entities seems odd, and indicates a possible bug in the design. That is
> another reason why using so-called namespaces while creating type
> hierarchies is a good practice.
>
> You could argue that if you have multiple sources for templates (e.g. your
> own storage and a external catalog), there could be unexpected name
> collisions of node types names. Having said that, we need to remember the
> entire process of the substitution mapping.
>
> First, as you said, we are trying to locate a service template that has a
> substitution mapping definition of the same type of the abstract node in
> the top level template. But choosing the service template is only the first
> step. The second step is instantiating the top level template along with
> the substitution template. During the instantiation, requirements and
> capabilities are matched. If not all requirements are satisfied
> (considering both top level and substituting template), the substitution
> mapping is not considered successful, and the instantiation fails.
>
> If that is the case, it makes sense to go ahead and try the next
> substitution template that fits the type, and continue like so until we
> find a substitution where all the requirements are satisfied during
> instantiation.
>
> At this point you may be wondering, "what if the instantiation was
> successful, but the source of the substitution template wasn't the one that
> *I* wanted?". Well, You have to think about this issue in substitution
> mapping terms. that is, when you use an abstract node, actually what you
> are saying is "I want something of type t, the has the capabilities c, and
> requires r. regarding the internals, I don't care, I just want it to work".
> So actually, selecting a specific substituting template (as long as the
> type, the requirements and the capabilities are as expected, of course), is
> somewhat against the spirit of the substitution mapping feature.
>
> 2) I'm sorry, but I didn't get to the bottom of your question. Could you
> elaborate a bit more, and include a small example?
>
>
>
>
> On Thu, Aug 31, 2017 at 11:35 AM, D Jayachandran <
> d.jayachand...@ericsson.com> wrote:
>
> > Hi,
> >
> > With respect to substituting stored service templates, I have few things
> > to be clarified
> >
> > 1) Handling substitution when multiple service templates matches for the
> > abstract node type .
> > Would the 1st match would be used for substitution  or Are we
> > looking at policy to enable user to select particular service templates
> for
> > substitution with multiple service templates ?
> > 2) Custom node types as abstract node type.
> > With custom node types as abstract node type, there seems to be a
> > need to implicitly import that custom node type in our top level service
> > template so that the parser recognizes this custom type.
> > Assuming the abstract node type would be substituted from a
> stored
> > substituting service template, we need to at least import the custom node
> > types and have it part of the same CSAR package.
> > Would this be a challenge for the top-level service template
> > author in including and importing the custom node types as abstraction ?
> or
> > Is this how we are looking at custom node types ?
> > Is it possible to identify an abstract node during parsing , such
> > as if it does not contain any implementation ( The SPEC does not say
> > anything on this )?
> >
> > Regards,
> > DJ
> > -Original Message-
> > Fr

Re: Service Composition / Substitution Mapping

[Avia Efrat]

1) Substitutable nodes (abstract nodes) are defined by their type. And a
node type should represent specific node/subsystem characteristics. Having
two node types ( with the same name) that represent different logical
entities seems odd, and indicates a possible bug in the design. That is
another reason why using so-called namespaces while creating type
hierarchies is a good practice.

You could argue that if you have multiple sources for templates (e.g. your
own storage and a external catalog), there could be unexpected name
collisions of node types names. Having said that, we need to remember the
entire process of the substitution mapping.

First, as you said, we are trying to locate a service template that has a
substitution mapping definition of the same type of the abstract node in
the top level template. But choosing the service template is only the first
step. The second step is instantiating the top level template along with
the substitution template. During the instantiation, requirements and
capabilities are matched. If not all requirements are satisfied
(considering both top level and substituting template), the substitution
mapping is not considered successful, and the instantiation fails.

If that is the case, it makes sense to go ahead and try the next
substitution template that fits the type, and continue like so until we
find a substitution where all the requirements are satisfied during
instantiation.

At this point you may be wondering, "what if the instantiation was
successful, but the source of the substitution template wasn't the one that
*I* wanted?". Well, You have to think about this issue in substitution
mapping terms. that is, when you use an abstract node, actually what you
are saying is "I want something of type t, the has the capabilities c, and
requires r. regarding the internals, I don't care, I just want it to work".
So actually, selecting a specific substituting template (as long as the
type, the requirements and the capabilities are as expected, of course), is
somewhat against the spirit of the substitution mapping feature.

2) I'm sorry, but I didn't get to the bottom of your question. Could you
elaborate a bit more, and include a small example?




On Thu, Aug 31, 2017 at 11:35 AM, D Jayachandran <
d.jayachand...@ericsson.com> wrote:

> Hi,
>
> With respect to substituting stored service templates, I have few things
> to be clarified
>
> 1) Handling substitution when multiple service templates matches for the
> abstract node type .
> Would the 1st match would be used for substitution  or Are we
> looking at policy to enable user to select particular service templates for
> substitution with multiple service templates ?
> 2) Custom node types as abstract node type.
> With custom node types as abstract node type, there seems to be a
> need to implicitly import that custom node type in our top level service
> template so that the parser recognizes this custom type.
> Assuming the abstract node type would be substituted from a stored
> substituting service template, we need to at least import the custom node
> types and have it part of the same CSAR package.
> Would this be a challenge for the top-level service template
> author in including and importing the custom node types as abstraction ? or
> Is this how we are looking at custom node types ?
> Is it possible to identify an abstract node during parsing , such
> as if it does not contain any implementation ( The SPEC does not say
> anything on this )?
>
> Regards,
> DJ
> -Original Message-
> From: Ran Ziv [mailto:r...@cloudify.co]
> Sent: Wednesday, August 16, 2017 6:19 PM
> To: dev@ariatosca.incubator.apache.org
> Subject: Re: Service Composition / Substitution Mapping
>
> I agree, especially when the benefit of being able to use an existing
> service - yet only one which hasn't been deployed via a workflow - doesn't
> seem all that interesting IMO.
>
> Another concern I could add to the ones you've mentioned is the service's
> inputs - the substituting template's inputs should be received via the
> properties of the abstract node in the top level service template. If the
> service already exists, these inputs would not be passed as expected.
>
> Ran
>
> On Wed, Aug 16, 2017 at 3:25 PM, D Jayachandran <
> d.jayachand...@ericsson.com
> > wrote:
>
> > Hi Ran,
> >
> > When Tal mentioned about "substituting service", I thought it was
> > about the services which dint have any associated executions/workflows
> triggered.
> > Am also in favor of  "substituting service templates" rather than
> > "substituting service".
> > With "substituting service" approach (when the service is not
> > instantiated), I

RE: Service Composition / Substitution Mapping

[D Jayachandran]

Hi,

With respect to substituting stored service templates, I have few things to be 
clarified

1) Handling substitution when multiple service templates matches for the 
abstract node type .
Would the 1st match would be used for substitution  or Are we looking 
at policy to enable user to select particular service templates for 
substitution with multiple service templates ?
2) Custom node types as abstract node type. 
With custom node types as abstract node type, there seems to be a need 
to implicitly import that custom node type in our top level service template so 
that the parser recognizes this custom type. 
Assuming the abstract node type would be substituted from a stored 
substituting service template, we need to at least import the custom node types 
and have it part of the same CSAR package. 
Would this be a challenge for the top-level service template author in 
including and importing the custom node types as abstraction ? or Is this how 
we are looking at custom node types ?
Is it possible to identify an abstract node during parsing , such as if 
it does not contain any implementation ( The SPEC does not say anything on this 
)? 

Regards,
DJ
-Original Message-
From: Ran Ziv [mailto:r...@cloudify.co] 
Sent: Wednesday, August 16, 2017 6:19 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping

I agree, especially when the benefit of being able to use an existing service - 
yet only one which hasn't been deployed via a workflow - doesn't seem all that 
interesting IMO.

Another concern I could add to the ones you've mentioned is the service's 
inputs - the substituting template's inputs should be received via the 
properties of the abstract node in the top level service template. If the 
service already exists, these inputs would not be passed as expected.

Ran

On Wed, Aug 16, 2017 at 3:25 PM, D Jayachandran <d.jayachand...@ericsson.com
> wrote:

> Hi Ran,
>
> When Tal mentioned about "substituting service", I thought it was 
> about the services which dint have any associated executions/workflows 
> triggered.
> Am also in favor of  "substituting service templates" rather than 
> "substituting service".
> With "substituting service" approach (when the service is not 
> instantiated), I see some open points
> - In a multi-user scenario, what will happen when a service is 
> composed using the substituting service and at the sametime a  
> workflow is triggered for the substituting service. ?
> - Is it okay to delete(dissolve) the substituting service 
> after it is used to create the composed service. ?
>
> Starting with it might be a good idea to only have "substituting 
> service templates" approach.
>
> Regards,
> DJ
> -Original Message-
> From: Ran Ziv [mailto:r...@cloudify.co]
> Sent: Wednesday, August 16, 2017 4:29 PM
> To: dev@ariatosca.incubator.apache.org
> Subject: Re: Service Composition / Substitution Mapping
>
> I'd say right now we're looking at "static service composition" which 
> is only about "substituting service templates", not "substituting 
> service". If a service is already running, it will not be used.
>
> I think what Tal meant was that each service template - whether the 
> top level one or one of the substituting templates - needs to resolve 
> its inner reqs internally first, and then resolve substitution 
> reqs across service templates.
>
>
> On Wed, Aug 16, 2017 at 12:00 PM, D Jayachandran < 
> d.jayachand...@ericsson.com> wrote:
>
> > Hi Tal,
> >
> > Thanks for organizing the points.
> > So if I understand correctly we are looking only at "Static service 
> > composition" which includes "substituting service template" and 
> > "substituting service".
> >
> > As you said with "substituting service template" approach ,we will 
> > have all the nodes aggregated from other service templates and a 
> > single workflow would be triggered to perform life-cycle operation 
> > on
> all the nodes.
> > Am not sure why the workflows needs to be "boundary aware" for nodes 
> > being substituted ? I see nodes are already part of the composed 
> > service, Could you please help me understand this ?
> >
> >
> > Regards,
> > DJ
> > -Original Message-
> > From: Tal Liron [mailto:t...@cloudify.co]
> > Sent: Saturday, August 12, 2017 4:52 AM
> > To: dev@ariatosca.incubator.apache.org
> > Subject: Re: Service Composition / Substitution Mapping
> >
> > You are correct -- to participate in this "multi-VIM" scenario, the 
> > Opensta

Re: Service Composition / Substitution Mapping

[Ran Ziv]

I agree, especially when the benefit of being able to use an existing
service - yet only one which hasn't been deployed via a workflow - doesn't
seem all that interesting IMO.

Another concern I could add to the ones you've mentioned is the service's
inputs - the substituting template's inputs should be received via the
properties of the abstract node in the top level service template. If the
service already exists, these inputs would not be passed as expected.

Ran

On Wed, Aug 16, 2017 at 3:25 PM, D Jayachandran <d.jayachand...@ericsson.com
> wrote:

> Hi Ran,
>
> When Tal mentioned about "substituting service", I thought it was about
> the services which dint have any associated executions/workflows triggered.
> Am also in favor of  "substituting service templates" rather than
> "substituting service".
> With "substituting service" approach (when the service is not
> instantiated), I see some open points
> - In a multi-user scenario, what will happen when a service is
> composed using the substituting service and at the sametime a  workflow is
> triggered for the substituting service. ?
> - Is it okay to delete(dissolve) the substituting service after it
> is used to create the composed service. ?
>
> Starting with it might be a good idea to only have "substituting service
> templates" approach.
>
> Regards,
> DJ
> -Original Message-
> From: Ran Ziv [mailto:r...@cloudify.co]
> Sent: Wednesday, August 16, 2017 4:29 PM
> To: dev@ariatosca.incubator.apache.org
> Subject: Re: Service Composition / Substitution Mapping
>
> I'd say right now we're looking at "static service composition" which is
> only about "substituting service templates", not "substituting service". If
> a service is already running, it will not be used.
>
> I think what Tal meant was that each service template - whether the top
> level one or one of the substituting templates - needs to resolve its inner
> reqs internally first, and then resolve substitution reqs across
> service templates.
>
>
> On Wed, Aug 16, 2017 at 12:00 PM, D Jayachandran <
> d.jayachand...@ericsson.com> wrote:
>
> > Hi Tal,
> >
> > Thanks for organizing the points.
> > So if I understand correctly we are looking only at "Static service
> > composition" which includes "substituting service template" and
> > "substituting service".
> >
> > As you said with "substituting service template" approach ,we will
> > have all the nodes aggregated from other service templates and a
> > single workflow would be triggered to perform life-cycle operation on
> all the nodes.
> > Am not sure why the workflows needs to be "boundary aware" for nodes
> > being substituted ? I see nodes are already part of the composed
> > service, Could you please help me understand this ?
> >
> >
> > Regards,
> > DJ
> > -Original Message-
> > From: Tal Liron [mailto:t...@cloudify.co]
> > Sent: Saturday, August 12, 2017 4:52 AM
> > To: dev@ariatosca.incubator.apache.org
> > Subject: Re: Service Composition / Substitution Mapping
> >
> > You are correct -- to participate in this "multi-VIM" scenario, the
> > Openstack plugin would have to know how to translate the TOSCA
> > properties to a flavor ID. This could all be done in 100% TOSCA via
> > policies (say, an aria.Openstack).
> >
> > Doing this automatically might not be a good idea, or even necessary.
> > Worst case is you get a validation error if the ARIA plugin can't find
> > a flavor in the table to match your requirements, in which you case
> > you can go and manually find the right ID and add it to the table.
> >
> > And I agree about being fine with imperfection: the rule of thumb for
> > our work is always to allow for sensible defaults even if no explicit
> > policy is given.
> >
> > Anyway, we've gone way off the topic of this thread. We can talk about
> > it more once it comes closer to an implementation.
> >
> > On Fri, Aug 11, 2017 at 3:52 PM, DeWayne Filppi <dewa...@cloudify.co>
> > wrote:
> >
> > > Interesting.  Take Openstack (please ).  If you model a
> > > compute OS as explicitly as you like, there still has to be a "match"
> > > to an Openstack image ID.  Or are you saying you must supply the
> > > image
> > ID for
> > > OSs.   Likewise, you can't supply RAM and CPUs without a flavor ID.
> > > Openstack does allow for custom flavors, but I doubt the plugin is
> > > doing that.  Much be

RE: Service Composition / Substitution Mapping

[D Jayachandran]

Hi Ran,

When Tal mentioned about "substituting service", I thought it was about the 
services which dint have any associated executions/workflows triggered.
Am also in favor of  "substituting service templates" rather than "substituting 
service".
With "substituting service" approach (when the service is not instantiated), I 
see some open points
- In a multi-user scenario, what will happen when a service is composed 
using the substituting service and at the sametime a  workflow is triggered for 
the substituting service. ?
- Is it okay to delete(dissolve) the substituting service after it is 
used to create the composed service. ? 

Starting with it might be a good idea to only have "substituting service 
templates" approach.

Regards,
DJ
-Original Message-
From: Ran Ziv [mailto:r...@cloudify.co] 
Sent: Wednesday, August 16, 2017 4:29 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping

I'd say right now we're looking at "static service composition" which is only 
about "substituting service templates", not "substituting service". If a 
service is already running, it will not be used.

I think what Tal meant was that each service template - whether the top level 
one or one of the substituting templates - needs to resolve its inner reqs 
internally first, and then resolve substitution reqs across service 
templates.


On Wed, Aug 16, 2017 at 12:00 PM, D Jayachandran < d.jayachand...@ericsson.com> 
wrote:

> Hi Tal,
>
> Thanks for organizing the points.
> So if I understand correctly we are looking only at "Static service 
> composition" which includes "substituting service template" and 
> "substituting service".
>
> As you said with "substituting service template" approach ,we will 
> have all the nodes aggregated from other service templates and a 
> single workflow would be triggered to perform life-cycle operation on all the 
> nodes.
> Am not sure why the workflows needs to be "boundary aware" for nodes 
> being substituted ? I see nodes are already part of the composed 
> service, Could you please help me understand this ?
>
>
> Regards,
> DJ
> -Original Message-
> From: Tal Liron [mailto:t...@cloudify.co]
> Sent: Saturday, August 12, 2017 4:52 AM
> To: dev@ariatosca.incubator.apache.org
> Subject: Re: Service Composition / Substitution Mapping
>
> You are correct -- to participate in this "multi-VIM" scenario, the 
> Openstack plugin would have to know how to translate the TOSCA 
> properties to a flavor ID. This could all be done in 100% TOSCA via 
> policies (say, an aria.Openstack).
>
> Doing this automatically might not be a good idea, or even necessary.
> Worst case is you get a validation error if the ARIA plugin can't find 
> a flavor in the table to match your requirements, in which you case 
> you can go and manually find the right ID and add it to the table.
>
> And I agree about being fine with imperfection: the rule of thumb for 
> our work is always to allow for sensible defaults even if no explicit 
> policy is given.
>
> Anyway, we've gone way off the topic of this thread. We can talk about 
> it more once it comes closer to an implementation.
>
> On Fri, Aug 11, 2017 at 3:52 PM, DeWayne Filppi <dewa...@cloudify.co>
> wrote:
>
> > Interesting.  Take Openstack (please ).  If you model a 
> > compute OS as explicitly as you like, there still has to be a "match"
> > to an Openstack image ID.  Or are you saying you must supply the 
> > image
> ID for
> > OSs.   Likewise, you can't supply RAM and CPUs without a flavor ID.
> > Openstack does allow for custom flavors, but I doubt the plugin is 
> > doing that.  Much better to have a "caps-init" interface in some low 
> > down base type that can be triggered at service creation to support 
> > reqs/caps (IMHO).  Then the plugin can verify whether the service 
> > can be construct based on fuzzy constraints.  Maybe this is a case 
> > that the "general solution" is a nightmare of complexity, but having 
> > a plugin scan the available flavors to make sure a requirement can 
> > be met doesn't seem that tough.  If TOSCA provided a formal 
> > lifecycle interface for it, then orchestrators or just plugins could 
> > determine how tricky they wanted to be.  IOW, let not the perfect be 
> > the enemy of
> the good.
> >
> > DeWayne
> >
> >
> > On Fri, Aug 11, 2017 at 1:26 PM, Tal Liron <t...@cloudify.co> wrote:
> >
> > > OK, that's a whole different can of worms. :)
> > >
> > > TOSCA's C

Re: Service Composition / Substitution Mapping

[Ran Ziv]

I'd say right now we're looking at "static service composition" which is
only about "substituting service templates", not "substituting service". If
a service is already running, it will not be used.

I think what Tal meant was that each service template - whether the top
level one or one of the substituting templates - needs to resolve its inner
reqs internally first, and then resolve substitution reqs across
service templates.


On Wed, Aug 16, 2017 at 12:00 PM, D Jayachandran <
d.jayachand...@ericsson.com> wrote:

> Hi Tal,
>
> Thanks for organizing the points.
> So if I understand correctly we are looking only at "Static service
> composition" which includes "substituting service template" and
> "substituting service".
>
> As you said with "substituting service template" approach ,we will have
> all the nodes aggregated from other service templates and a single workflow
> would be triggered to perform life-cycle operation on all the nodes.
> Am not sure why the workflows needs to be "boundary aware" for nodes being
> substituted ? I see nodes are already part of the composed service, Could
> you please help me understand this ?
>
>
> Regards,
> DJ
> -Original Message-
> From: Tal Liron [mailto:t...@cloudify.co]
> Sent: Saturday, August 12, 2017 4:52 AM
> To: dev@ariatosca.incubator.apache.org
> Subject: Re: Service Composition / Substitution Mapping
>
> You are correct -- to participate in this "multi-VIM" scenario, the
> Openstack plugin would have to know how to translate the TOSCA properties
> to a flavor ID. This could all be done in 100% TOSCA via policies (say, an
> aria.Openstack).
>
> Doing this automatically might not be a good idea, or even necessary.
> Worst case is you get a validation error if the ARIA plugin can't find a
> flavor in the table to match your requirements, in which you case you can
> go and manually find the right ID and add it to the table.
>
> And I agree about being fine with imperfection: the rule of thumb for our
> work is always to allow for sensible defaults even if no explicit policy is
> given.
>
> Anyway, we've gone way off the topic of this thread. We can talk about it
> more once it comes closer to an implementation.
>
> On Fri, Aug 11, 2017 at 3:52 PM, DeWayne Filppi <dewa...@cloudify.co>
> wrote:
>
> > Interesting.  Take Openstack (please ).  If you model a
> > compute OS as explicitly as you like, there still has to be a "match"
> > to an Openstack image ID.  Or are you saying you must supply the image
> ID for
> > OSs.   Likewise, you can't supply RAM and CPUs without a flavor ID.
> > Openstack does allow for custom flavors, but I doubt the plugin is
> > doing that.  Much better to have a "caps-init" interface in some low
> > down base type that can be triggered at service creation to support
> > reqs/caps (IMHO).  Then the plugin can verify whether the service can
> > be construct based on fuzzy constraints.  Maybe this is a case that
> > the "general solution" is a nightmare of complexity, but having a
> > plugin scan the available flavors to make sure a requirement can be
> > met doesn't seem that tough.  If TOSCA provided a formal lifecycle
> > interface for it, then orchestrators or just plugins could determine
> > how tricky they wanted to be.  IOW, let not the perfect be the enemy of
> the good.
> >
> > DeWayne
> >
> >
> > On Fri, Aug 11, 2017 at 1:26 PM, Tal Liron <t...@cloudify.co> wrote:
> >
> > > OK, that's a whole different can of worms. :)
> > >
> > > TOSCA's Compute capabilities (Container and OperatingSystem) are
> > explicit.
> > > You specify which OS you want, how much RAM you want, how many CPUs,
> etc.
> > > ARIA's explicit node types (for example, the AWS Compute node) are
> > likewise
> > > explicit. So there is not querying here: the plugin will attempt to
> > > spin
> > up
> > > exactly the virtual machine you asked for. If it fails, the workflow
> > > will fail.
> > >
> > > This is not good enough, I think, for real world scenarios. There
> > > are two possible solutions:
> > >
> > > 1) Support ranges or fallbacks. So instead of saying "I need 4 GB of
> RAM"
> > > you can say "I would like 4 GB of RAM, but 2 GB would also be OK."
> > There's
> > > no easy way to do this now without totally changing how the
> > > capability types are designed. But, it may be possible to override
> > > this via
> > policies.
> > > So,

Re: Service Composition / Substitution Mapping

[Ran Ziv]

I'd say right now we're looking at "static service composition" which is
only about "substituting service templates", not "substituting service". If
a service is already running, it will not be used.

I think what Tal meant was that each service template - whether the top
level one or one of the substituting templates - needs to resolve its inner
reqs internally first, and then resolve substitution reqs across
service templates.


On Wed, Aug 16, 2017 at 12:00 PM, D Jayachandran <
d.jayachand...@ericsson.com> wrote:

> Hi Tal,
>
> Thanks for organizing the points.
> So if I understand correctly we are looking only at "Static service
> composition" which includes "substituting service template" and
> "substituting service".
>
> As you said with "substituting service template" approach ,we will have
> all the nodes aggregated from other service templates and a single workflow
> would be triggered to perform life-cycle operation on all the nodes.
> Am not sure why the workflows needs to be "boundary aware" for nodes being
> substituted ? I see nodes are already part of the composed service, Could
> you please help me understand this ?
>
>
> Regards,
> DJ
> -Original Message-
> From: Tal Liron [mailto:t...@cloudify.co]
> Sent: Saturday, August 12, 2017 4:52 AM
> To: dev@ariatosca.incubator.apache.org
> Subject: Re: Service Composition / Substitution Mapping
>
> You are correct -- to participate in this "multi-VIM" scenario, the
> Openstack plugin would have to know how to translate the TOSCA properties
> to a flavor ID. This could all be done in 100% TOSCA via policies (say, an
> aria.Openstack).
>
> Doing this automatically might not be a good idea, or even necessary.
> Worst case is you get a validation error if the ARIA plugin can't find a
> flavor in the table to match your requirements, in which you case you can
> go and manually find the right ID and add it to the table.
>
> And I agree about being fine with imperfection: the rule of thumb for our
> work is always to allow for sensible defaults even if no explicit policy is
> given.
>
> Anyway, we've gone way off the topic of this thread. We can talk about it
> more once it comes closer to an implementation.
>
> On Fri, Aug 11, 2017 at 3:52 PM, DeWayne Filppi <dewa...@cloudify.co>
> wrote:
>
> > Interesting.  Take Openstack (please ).  If you model a
> > compute OS as explicitly as you like, there still has to be a "match"
> > to an Openstack image ID.  Or are you saying you must supply the image
> ID for
> > OSs.   Likewise, you can't supply RAM and CPUs without a flavor ID.
> > Openstack does allow for custom flavors, but I doubt the plugin is
> > doing that.  Much better to have a "caps-init" interface in some low
> > down base type that can be triggered at service creation to support
> > reqs/caps (IMHO).  Then the plugin can verify whether the service can
> > be construct based on fuzzy constraints.  Maybe this is a case that
> > the "general solution" is a nightmare of complexity, but having a
> > plugin scan the available flavors to make sure a requirement can be
> > met doesn't seem that tough.  If TOSCA provided a formal lifecycle
> > interface for it, then orchestrators or just plugins could determine
> > how tricky they wanted to be.  IOW, let not the perfect be the enemy of
> the good.
> >
> > DeWayne
> >
> >
> > On Fri, Aug 11, 2017 at 1:26 PM, Tal Liron <t...@cloudify.co> wrote:
> >
> > > OK, that's a whole different can of worms. :)
> > >
> > > TOSCA's Compute capabilities (Container and OperatingSystem) are
> > explicit.
> > > You specify which OS you want, how much RAM you want, how many CPUs,
> etc.
> > > ARIA's explicit node types (for example, the AWS Compute node) are
> > likewise
> > > explicit. So there is not querying here: the plugin will attempt to
> > > spin
> > up
> > > exactly the virtual machine you asked for. If it fails, the workflow
> > > will fail.
> > >
> > > This is not good enough, I think, for real world scenarios. There
> > > are two possible solutions:
> > >
> > > 1) Support ranges or fallbacks. So instead of saying "I need 4 GB of
> RAM"
> > > you can say "I would like 4 GB of RAM, but 2 GB would also be OK."
> > There's
> > > no easy way to do this now without totally changing how the
> > > capability types are designed. But, it may be possible to override
> > > this via
> > policies.
> > > So,

RE: Service Composition / Substitution Mapping

[D Jayachandran]

Hi Tal,

Thanks for organizing the points.
So if I understand correctly we are looking only at "Static service 
composition" which includes "substituting service template" and "substituting 
service".

As you said with "substituting service template" approach ,we will have all the 
nodes aggregated from other service templates and a single workflow would be 
triggered to perform life-cycle operation on all the nodes.
Am not sure why the workflows needs to be "boundary aware" for nodes being 
substituted ? I see nodes are already part of the composed service, Could you 
please help me understand this ?


Regards,
DJ
-Original Message-
From: Tal Liron [mailto:t...@cloudify.co] 
Sent: Saturday, August 12, 2017 4:52 AM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping

You are correct -- to participate in this "multi-VIM" scenario, the Openstack 
plugin would have to know how to translate the TOSCA properties to a flavor ID. 
This could all be done in 100% TOSCA via policies (say, an aria.Openstack).

Doing this automatically might not be a good idea, or even necessary. Worst 
case is you get a validation error if the ARIA plugin can't find a flavor in 
the table to match your requirements, in which you case you can go and manually 
find the right ID and add it to the table.

And I agree about being fine with imperfection: the rule of thumb for our work 
is always to allow for sensible defaults even if no explicit policy is given.

Anyway, we've gone way off the topic of this thread. We can talk about it more 
once it comes closer to an implementation.

On Fri, Aug 11, 2017 at 3:52 PM, DeWayne Filppi <dewa...@cloudify.co> wrote:

> Interesting.  Take Openstack (please ).  If you model a 
> compute OS as explicitly as you like, there still has to be a "match" 
> to an Openstack image ID.  Or are you saying you must supply the image ID for
> OSs.   Likewise, you can't supply RAM and CPUs without a flavor ID.
> Openstack does allow for custom flavors, but I doubt the plugin is 
> doing that.  Much better to have a "caps-init" interface in some low 
> down base type that can be triggered at service creation to support 
> reqs/caps (IMHO).  Then the plugin can verify whether the service can 
> be construct based on fuzzy constraints.  Maybe this is a case that 
> the "general solution" is a nightmare of complexity, but having a 
> plugin scan the available flavors to make sure a requirement can be 
> met doesn't seem that tough.  If TOSCA provided a formal lifecycle 
> interface for it, then orchestrators or just plugins could determine 
> how tricky they wanted to be.  IOW, let not the perfect be the enemy of the 
> good.
>
> DeWayne
>
>
> On Fri, Aug 11, 2017 at 1:26 PM, Tal Liron <t...@cloudify.co> wrote:
>
> > OK, that's a whole different can of worms. :)
> >
> > TOSCA's Compute capabilities (Container and OperatingSystem) are
> explicit.
> > You specify which OS you want, how much RAM you want, how many CPUs, etc.
> > ARIA's explicit node types (for example, the AWS Compute node) are
> likewise
> > explicit. So there is not querying here: the plugin will attempt to 
> > spin
> up
> > exactly the virtual machine you asked for. If it fails, the workflow 
> > will fail.
> >
> > This is not good enough, I think, for real world scenarios. There 
> > are two possible solutions:
> >
> > 1) Support ranges or fallbacks. So instead of saying "I need 4 GB of RAM"
> > you can say "I would like 4 GB of RAM, but 2 GB would also be OK."
> There's
> > no easy way to do this now without totally changing how the 
> > capability types are designed. But, it may be possible to override 
> > this via
> policies.
> > So, the capabilities would perhaps specify the minimal requirements,
> while
> > policies specify preferences. Some aspects of this were discussed in 
> > the OPEN-O project. DeWayne, has any of this survived in ONAP, or 
> > have we not reached that point in the discussion yet?
> >
> > 2) Incorporate this into the bigger topic of resource orchestration.
> This a
> > huge challenge for the industry. The problem field contains not just 
> > "I need X amount of RAM" but also "I want all my virtual machines 
> > and containers running on the same high-performance network backend 
> > and have these two nodes on the same bare-metal machine or at least 
> > in the same
> data
> > center rack with a NUMA interconnect, and I also don't want all this 
> > to cost more that $100 per hour." That's not crazy: these are real 
> > world requirements for hi

Re: Service Composition / Substitution Mapping

[Tal Liron]

You are correct -- to participate in this "multi-VIM" scenario, the
Openstack plugin would have to know how to translate the TOSCA properties
to a flavor ID. This could all be done in 100% TOSCA via policies (say, an
aria.Openstack).

Doing this automatically might not be a good idea, or even necessary. Worst
case is you get a validation error if the ARIA plugin can't find a flavor
in the table to match your requirements, in which you case you can go and
manually find the right ID and add it to the table.

And I agree about being fine with imperfection: the rule of thumb for our
work is always to allow for sensible defaults even if no explicit policy is
given.

Anyway, we've gone way off the topic of this thread. We can talk about it
more once it comes closer to an implementation.

On Fri, Aug 11, 2017 at 3:52 PM, DeWayne Filppi  wrote:

> Interesting.  Take Openstack (please ).  If you model a compute
> OS as explicitly as you like, there still has to be a "match" to an
> Openstack image ID.  Or are you saying you must supply the image ID for
> OSs.   Likewise, you can't supply RAM and CPUs without a flavor ID.
> Openstack does allow for custom flavors, but I doubt the plugin is doing
> that.  Much better to have a "caps-init" interface in some low down base
> type that can be triggered at service creation to support reqs/caps
> (IMHO).  Then the plugin can verify whether the service can be construct
> based on fuzzy constraints.  Maybe this is a case that the "general
> solution" is a nightmare of complexity, but having a plugin scan the
> available flavors to make sure a requirement can be met doesn't seem that
> tough.  If TOSCA provided a formal lifecycle interface for it, then
> orchestrators or just plugins could determine how tricky they wanted to
> be.  IOW, let not the perfect be the enemy of the good.
>
> DeWayne
>
>
> On Fri, Aug 11, 2017 at 1:26 PM, Tal Liron  wrote:
>
> > OK, that's a whole different can of worms. :)
> >
> > TOSCA's Compute capabilities (Container and OperatingSystem) are
> explicit.
> > You specify which OS you want, how much RAM you want, how many CPUs, etc.
> > ARIA's explicit node types (for example, the AWS Compute node) are
> likewise
> > explicit. So there is not querying here: the plugin will attempt to spin
> up
> > exactly the virtual machine you asked for. If it fails, the workflow will
> > fail.
> >
> > This is not good enough, I think, for real world scenarios. There are two
> > possible solutions:
> >
> > 1) Support ranges or fallbacks. So instead of saying "I need 4 GB of RAM"
> > you can say "I would like 4 GB of RAM, but 2 GB would also be OK."
> There's
> > no easy way to do this now without totally changing how the capability
> > types are designed. But, it may be possible to override this via
> policies.
> > So, the capabilities would perhaps specify the minimal requirements,
> while
> > policies specify preferences. Some aspects of this were discussed in the
> > OPEN-O project. DeWayne, has any of this survived in ONAP, or have we not
> > reached that point in the discussion yet?
> >
> > 2) Incorporate this into the bigger topic of resource orchestration.
> This a
> > huge challenge for the industry. The problem field contains not just "I
> > need X amount of RAM" but also "I want all my virtual machines and
> > containers running on the same high-performance network backend and have
> > these two nodes on the same bare-metal machine or at least in the same
> data
> > center rack with a NUMA interconnect, and I also don't want all this to
> > cost more that $100 per hour." That's not crazy: these are real world
> > requirements for high-performance VNFs and service chaining. Resource
> > orchestration requires a full map of what is available in the data
> centers,
> > a negotiation-based algorithm for properly allocating and placing
> > resources, connection to billing services, etc. Of course resource
> > orchestration is not within the scope of ARIA, but it would be great for
> > ARIA to have plugins for them (and TOSCA be able to model resource
> > requirement policies) when these become available.
> >
> >
> >
> > On Fri, Aug 11, 2017 at 3:02 PM, DeWayne Filppi 
> > wrote:
> >
> > > For the "resource realization" part, I was not even considering
> > > multicloud/vim.   I was considering single cloud even outside of
> > > composition.  Just reqs and caps.  If my node "requires" a compute node
> > > with Suse Linux version X with a minimum of 4GB RAM, how does the
> > > orchestrator match that without querying the cloud via the plugin?   If
> > it
> > > does query it, which it seems it must in addition to the implicit quota
> > > query, how is this done?  TOSCA seems to not really care, which is fine
> > and
> > > perhaps a good idea.  But ARIA has to care.
> > >
> > > DeWayne
> > >
> > > On Fri, Aug 11, 2017 at 12:34 PM, Tal Liron  wrote:
> > >
> > > > In my opinion, the new 

Re: Service Composition / Substitution Mapping

[DeWayne Filppi]

Interesting.  Take Openstack (please ).  If you model a compute
OS as explicitly as you like, there still has to be a "match" to an
Openstack image ID.  Or are you saying you must supply the image ID for
OSs.   Likewise, you can't supply RAM and CPUs without a flavor ID.
Openstack does allow for custom flavors, but I doubt the plugin is doing
that.  Much better to have a "caps-init" interface in some low down base
type that can be triggered at service creation to support reqs/caps
(IMHO).  Then the plugin can verify whether the service can be construct
based on fuzzy constraints.  Maybe this is a case that the "general
solution" is a nightmare of complexity, but having a plugin scan the
available flavors to make sure a requirement can be met doesn't seem that
tough.  If TOSCA provided a formal lifecycle interface for it, then
orchestrators or just plugins could determine how tricky they wanted to
be.  IOW, let not the perfect be the enemy of the good.

DeWayne


On Fri, Aug 11, 2017 at 1:26 PM, Tal Liron  wrote:

> OK, that's a whole different can of worms. :)
>
> TOSCA's Compute capabilities (Container and OperatingSystem) are explicit.
> You specify which OS you want, how much RAM you want, how many CPUs, etc.
> ARIA's explicit node types (for example, the AWS Compute node) are likewise
> explicit. So there is not querying here: the plugin will attempt to spin up
> exactly the virtual machine you asked for. If it fails, the workflow will
> fail.
>
> This is not good enough, I think, for real world scenarios. There are two
> possible solutions:
>
> 1) Support ranges or fallbacks. So instead of saying "I need 4 GB of RAM"
> you can say "I would like 4 GB of RAM, but 2 GB would also be OK." There's
> no easy way to do this now without totally changing how the capability
> types are designed. But, it may be possible to override this via policies.
> So, the capabilities would perhaps specify the minimal requirements, while
> policies specify preferences. Some aspects of this were discussed in the
> OPEN-O project. DeWayne, has any of this survived in ONAP, or have we not
> reached that point in the discussion yet?
>
> 2) Incorporate this into the bigger topic of resource orchestration. This a
> huge challenge for the industry. The problem field contains not just "I
> need X amount of RAM" but also "I want all my virtual machines and
> containers running on the same high-performance network backend and have
> these two nodes on the same bare-metal machine or at least in the same data
> center rack with a NUMA interconnect, and I also don't want all this to
> cost more that $100 per hour." That's not crazy: these are real world
> requirements for high-performance VNFs and service chaining. Resource
> orchestration requires a full map of what is available in the data centers,
> a negotiation-based algorithm for properly allocating and placing
> resources, connection to billing services, etc. Of course resource
> orchestration is not within the scope of ARIA, but it would be great for
> ARIA to have plugins for them (and TOSCA be able to model resource
> requirement policies) when these become available.
>
>
>
> On Fri, Aug 11, 2017 at 3:02 PM, DeWayne Filppi 
> wrote:
>
> > For the "resource realization" part, I was not even considering
> > multicloud/vim.   I was considering single cloud even outside of
> > composition.  Just reqs and caps.  If my node "requires" a compute node
> > with Suse Linux version X with a minimum of 4GB RAM, how does the
> > orchestrator match that without querying the cloud via the plugin?   If
> it
> > does query it, which it seems it must in addition to the implicit quota
> > query, how is this done?  TOSCA seems to not really care, which is fine
> and
> > perhaps a good idea.  But ARIA has to care.
> >
> > DeWayne
> >
> > On Fri, Aug 11, 2017 at 12:34 PM, Tal Liron  wrote:
> >
> > > In my opinion, the new composite service should indeed be a single
> > service,
> > > so the ARIA CLI will show it as one (if a substituting service already
> > > existed, it would be "dissolved" into the new top-level service). The
> > > composition will show its traces if you look more deeply, because
> you'll
> > > see that some node templates come from a different service template.
> > > Perhaps our CLI can detect this and tell the user which service
> templates
> > > were used to create the nodes in current service.
> > >
> > > The rest of what you described I think is not related directly to
> service
> > > composition, but resource realization. The way we currently handle this
> > is
> > > by being very explicit: you have to use the derived Compute types
> > included
> > > in our AWS or Openstack plugins, for example.
> > >
> > > I think we should still keep this, because sometimes you want to be
> very
> > > explicit (and use specific features of the platform), but I actually
> have
> > > some ideas for multi-VIM support that would work 

Re: Service Composition / Substitution Mapping

[Tal Liron]

OK, that's a whole different can of worms. :)

TOSCA's Compute capabilities (Container and OperatingSystem) are explicit.
You specify which OS you want, how much RAM you want, how many CPUs, etc.
ARIA's explicit node types (for example, the AWS Compute node) are likewise
explicit. So there is not querying here: the plugin will attempt to spin up
exactly the virtual machine you asked for. If it fails, the workflow will
fail.

This is not good enough, I think, for real world scenarios. There are two
possible solutions:

1) Support ranges or fallbacks. So instead of saying "I need 4 GB of RAM"
you can say "I would like 4 GB of RAM, but 2 GB would also be OK." There's
no easy way to do this now without totally changing how the capability
types are designed. But, it may be possible to override this via policies.
So, the capabilities would perhaps specify the minimal requirements, while
policies specify preferences. Some aspects of this were discussed in the
OPEN-O project. DeWayne, has any of this survived in ONAP, or have we not
reached that point in the discussion yet?

2) Incorporate this into the bigger topic of resource orchestration. This a
huge challenge for the industry. The problem field contains not just "I
need X amount of RAM" but also "I want all my virtual machines and
containers running on the same high-performance network backend and have
these two nodes on the same bare-metal machine or at least in the same data
center rack with a NUMA interconnect, and I also don't want all this to
cost more that $100 per hour." That's not crazy: these are real world
requirements for high-performance VNFs and service chaining. Resource
orchestration requires a full map of what is available in the data centers,
a negotiation-based algorithm for properly allocating and placing
resources, connection to billing services, etc. Of course resource
orchestration is not within the scope of ARIA, but it would be great for
ARIA to have plugins for them (and TOSCA be able to model resource
requirement policies) when these become available.



On Fri, Aug 11, 2017 at 3:02 PM, DeWayne Filppi  wrote:

> For the "resource realization" part, I was not even considering
> multicloud/vim.   I was considering single cloud even outside of
> composition.  Just reqs and caps.  If my node "requires" a compute node
> with Suse Linux version X with a minimum of 4GB RAM, how does the
> orchestrator match that without querying the cloud via the plugin?   If it
> does query it, which it seems it must in addition to the implicit quota
> query, how is this done?  TOSCA seems to not really care, which is fine and
> perhaps a good idea.  But ARIA has to care.
>
> DeWayne
>
> On Fri, Aug 11, 2017 at 12:34 PM, Tal Liron  wrote:
>
> > In my opinion, the new composite service should indeed be a single
> service,
> > so the ARIA CLI will show it as one (if a substituting service already
> > existed, it would be "dissolved" into the new top-level service). The
> > composition will show its traces if you look more deeply, because you'll
> > see that some node templates come from a different service template.
> > Perhaps our CLI can detect this and tell the user which service templates
> > were used to create the nodes in current service.
> >
> > The rest of what you described I think is not related directly to service
> > composition, but resource realization. The way we currently handle this
> is
> > by being very explicit: you have to use the derived Compute types
> included
> > in our AWS or Openstack plugins, for example.
> >
> > I think we should still keep this, because sometimes you want to be very
> > explicit (and use specific features of the platform), but I actually have
> > some ideas for multi-VIM support that would work differently: the idea is
> > to use the basic Compute type, with detailed properties in its Container
> > and OperatingSystem capabilities. There would then be a policy, perhaps
> > aria.Realization, that would hint at what kind of Compute nodes you are
> > looking for (you can apply the policy to specific nodes or groups, or
> > generally). It might be possible to still have strong platform-specific
> > feature support here, perhaps by implementing a general string key-value
> > dict that could include hints for the specific plugin to use.
> >
> > Anyway, just ideas at this point...
> >
> >
> > On Fri, Aug 11, 2017 at 2:25 PM, DeWayne Filppi 
> > wrote:
> >
> > > Good stuff.  Obviously a "fancy include" was the wrong metaphor.  The
> > > lifecycles are linked though.  When a create the referencing service
> > (aria
> > > service create), ARIA will match the other service template, and run
> > create
> > > service on it, and then continue on.  Uninstall of the referencer will
> > > destroy the referenced service.  After such an instantiation, would
> > listing
> > > all services produce two (perhaps with some parent indicator) or one?
> > From
> > > what I've 

Re: Service Composition / Substitution Mapping

[DeWayne Filppi]

For the "resource realization" part, I was not even considering
multicloud/vim.   I was considering single cloud even outside of
composition.  Just reqs and caps.  If my node "requires" a compute node
with Suse Linux version X with a minimum of 4GB RAM, how does the
orchestrator match that without querying the cloud via the plugin?   If it
does query it, which it seems it must in addition to the implicit quota
query, how is this done?  TOSCA seems to not really care, which is fine and
perhaps a good idea.  But ARIA has to care.

DeWayne

On Fri, Aug 11, 2017 at 12:34 PM, Tal Liron  wrote:

> In my opinion, the new composite service should indeed be a single service,
> so the ARIA CLI will show it as one (if a substituting service already
> existed, it would be "dissolved" into the new top-level service). The
> composition will show its traces if you look more deeply, because you'll
> see that some node templates come from a different service template.
> Perhaps our CLI can detect this and tell the user which service templates
> were used to create the nodes in current service.
>
> The rest of what you described I think is not related directly to service
> composition, but resource realization. The way we currently handle this is
> by being very explicit: you have to use the derived Compute types included
> in our AWS or Openstack plugins, for example.
>
> I think we should still keep this, because sometimes you want to be very
> explicit (and use specific features of the platform), but I actually have
> some ideas for multi-VIM support that would work differently: the idea is
> to use the basic Compute type, with detailed properties in its Container
> and OperatingSystem capabilities. There would then be a policy, perhaps
> aria.Realization, that would hint at what kind of Compute nodes you are
> looking for (you can apply the policy to specific nodes or groups, or
> generally). It might be possible to still have strong platform-specific
> feature support here, perhaps by implementing a general string key-value
> dict that could include hints for the specific plugin to use.
>
> Anyway, just ideas at this point...
>
>
> On Fri, Aug 11, 2017 at 2:25 PM, DeWayne Filppi 
> wrote:
>
> > Good stuff.  Obviously a "fancy include" was the wrong metaphor.  The
> > lifecycles are linked though.  When a create the referencing service
> (aria
> > service create), ARIA will match the other service template, and run
> create
> > service on it, and then continue on.  Uninstall of the referencer will
> > destroy the referenced service.  After such an instantiation, would
> listing
> > all services produce two (perhaps with some parent indicator) or one?
> From
> > what I've seen, reqs and caps don't have the equivalent of a lifecycle
> > interface associated.  Maybe I missed it.  It is implicit in the
> > orchestrator to match stuff up at service creation time, I'm assuming,
> and
> > using plugins somehow?  The sort of flagship scenario is matching cloud
> > compute resources without specifying an image.  Since discovering
> whether a
> > particular cloud has a given capability, I guessing a plugin for that
> cloud
> > has to search cloud inventory and perhaps quota to provide info for the
> > match.  Since plugins (at least the Cloudify variety) tend to be
> triggered
> > due to operations that they connect to lifecycle methods, and there is no
> > "materialize_capabilities" interface that I noticed, some other kind of
> > magic must be performed.  Maybe this is an orchestrator-specific detail
> > outside of TOSCA.
> >
> > DeWayne
> >
> > On Fri, Aug 11, 2017 at 10:57 AM, Tal Liron  wrote:
> >
> > > OK, let me try to organize this differently. Three potential
> conceptions
> > > here:
> > >
> > > 1) A "fancy include," as you say. All that would happen here is that
> the
> > > TOSCA parser would automatically find the service template to include
> > from
> > > some kind of repository of recognized service templates, and just
> include
> > > that. The language in the TOSCA spec suggests that this is not the
> > > intention: it is something that happens at the "orchestrator," not the
> > > "parser."
> > >
> > > 2) Static service composition. This happens not at the parsing stage,
> but
> > > rather the instantiation stage, where reqs-and-caps happens. I think
> this
> > > is what is intended: substitution mapping is specifically about mapping
> > > reqs-and-caps. And this is also where things like scalability and
> > placement
> > > happen: think of a requirement matching a capability, but that
> capability
> > > not having enough capacity. So, the node might be able to scale out: in
> > the
> > > case of substitution, this would mean duplicating an entire service
> > > instance. My understanding is that this is what is intended by TOSCA,
> and
> > > it's entirely within the scope of what we can do. We've recently just
> > > refactored the instantiation phase into a 

Re: Service Composition / Substitution Mapping

[Tal Liron]

In my opinion, the new composite service should indeed be a single service,
so the ARIA CLI will show it as one (if a substituting service already
existed, it would be "dissolved" into the new top-level service). The
composition will show its traces if you look more deeply, because you'll
see that some node templates come from a different service template.
Perhaps our CLI can detect this and tell the user which service templates
were used to create the nodes in current service.

The rest of what you described I think is not related directly to service
composition, but resource realization. The way we currently handle this is
by being very explicit: you have to use the derived Compute types included
in our AWS or Openstack plugins, for example.

I think we should still keep this, because sometimes you want to be very
explicit (and use specific features of the platform), but I actually have
some ideas for multi-VIM support that would work differently: the idea is
to use the basic Compute type, with detailed properties in its Container
and OperatingSystem capabilities. There would then be a policy, perhaps
aria.Realization, that would hint at what kind of Compute nodes you are
looking for (you can apply the policy to specific nodes or groups, or
generally). It might be possible to still have strong platform-specific
feature support here, perhaps by implementing a general string key-value
dict that could include hints for the specific plugin to use.

Anyway, just ideas at this point...


On Fri, Aug 11, 2017 at 2:25 PM, DeWayne Filppi  wrote:

> Good stuff.  Obviously a "fancy include" was the wrong metaphor.  The
> lifecycles are linked though.  When a create the referencing service (aria
> service create), ARIA will match the other service template, and run create
> service on it, and then continue on.  Uninstall of the referencer will
> destroy the referenced service.  After such an instantiation, would listing
> all services produce two (perhaps with some parent indicator) or one?  From
> what I've seen, reqs and caps don't have the equivalent of a lifecycle
> interface associated.  Maybe I missed it.  It is implicit in the
> orchestrator to match stuff up at service creation time, I'm assuming, and
> using plugins somehow?  The sort of flagship scenario is matching cloud
> compute resources without specifying an image.  Since discovering whether a
> particular cloud has a given capability, I guessing a plugin for that cloud
> has to search cloud inventory and perhaps quota to provide info for the
> match.  Since plugins (at least the Cloudify variety) tend to be triggered
> due to operations that they connect to lifecycle methods, and there is no
> "materialize_capabilities" interface that I noticed, some other kind of
> magic must be performed.  Maybe this is an orchestrator-specific detail
> outside of TOSCA.
>
> DeWayne
>
> On Fri, Aug 11, 2017 at 10:57 AM, Tal Liron  wrote:
>
> > OK, let me try to organize this differently. Three potential conceptions
> > here:
> >
> > 1) A "fancy include," as you say. All that would happen here is that the
> > TOSCA parser would automatically find the service template to include
> from
> > some kind of repository of recognized service templates, and just include
> > that. The language in the TOSCA spec suggests that this is not the
> > intention: it is something that happens at the "orchestrator," not the
> > "parser."
> >
> > 2) Static service composition. This happens not at the parsing stage, but
> > rather the instantiation stage, where reqs-and-caps happens. I think this
> > is what is intended: substitution mapping is specifically about mapping
> > reqs-and-caps. And this is also where things like scalability and
> placement
> > happen: think of a requirement matching a capability, but that capability
> > not having enough capacity. So, the node might be able to scale out: in
> the
> > case of substitution, this would mean duplicating an entire service
> > instance. My understanding is that this is what is intended by TOSCA, and
> > it's entirely within the scope of what we can do. We've recently just
> > refactored the instantiation phase into a new "topology" module where
> > exactly all this logic is concentrated. So think of it this way -- ARIA
> has
> > three parts: "parser," "topology manager," and "workflow engine"
> > ("orchestrator").
> >
> > (I think I might have confused some thing here a bit when mentioning a
> > "logical proxy node." I do not mean an actual piece of proxy software
> > running on a machine somewhere. I mean just a data point in the
> > ARIA-generated topology that can be used as a barrier of sorts when
> > constructing the task graph -- because the task graph follows
> > relationships, the edges of the topology. It could be that we discover in
> > our POC that this is not needed, because actually the substitution node
> is
> > already part of our topology data model and we might be able to 

Re: Service Composition / Substitution Mapping

[DeWayne Filppi]

Good stuff.  Obviously a "fancy include" was the wrong metaphor.  The
lifecycles are linked though.  When a create the referencing service (aria
service create), ARIA will match the other service template, and run create
service on it, and then continue on.  Uninstall of the referencer will
destroy the referenced service.  After such an instantiation, would listing
all services produce two (perhaps with some parent indicator) or one?  From
what I've seen, reqs and caps don't have the equivalent of a lifecycle
interface associated.  Maybe I missed it.  It is implicit in the
orchestrator to match stuff up at service creation time, I'm assuming, and
using plugins somehow?  The sort of flagship scenario is matching cloud
compute resources without specifying an image.  Since discovering whether a
particular cloud has a given capability, I guessing a plugin for that cloud
has to search cloud inventory and perhaps quota to provide info for the
match.  Since plugins (at least the Cloudify variety) tend to be triggered
due to operations that they connect to lifecycle methods, and there is no
"materialize_capabilities" interface that I noticed, some other kind of
magic must be performed.  Maybe this is an orchestrator-specific detail
outside of TOSCA.

DeWayne

On Fri, Aug 11, 2017 at 10:57 AM, Tal Liron  wrote:

> OK, let me try to organize this differently. Three potential conceptions
> here:
>
> 1) A "fancy include," as you say. All that would happen here is that the
> TOSCA parser would automatically find the service template to include from
> some kind of repository of recognized service templates, and just include
> that. The language in the TOSCA spec suggests that this is not the
> intention: it is something that happens at the "orchestrator," not the
> "parser."
>
> 2) Static service composition. This happens not at the parsing stage, but
> rather the instantiation stage, where reqs-and-caps happens. I think this
> is what is intended: substitution mapping is specifically about mapping
> reqs-and-caps. And this is also where things like scalability and placement
> happen: think of a requirement matching a capability, but that capability
> not having enough capacity. So, the node might be able to scale out: in the
> case of substitution, this would mean duplicating an entire service
> instance. My understanding is that this is what is intended by TOSCA, and
> it's entirely within the scope of what we can do. We've recently just
> refactored the instantiation phase into a new "topology" module where
> exactly all this logic is concentrated. So think of it this way -- ARIA has
> three parts: "parser," "topology manager," and "workflow engine"
> ("orchestrator").
>
> (I think I might have confused some thing here a bit when mentioning a
> "logical proxy node." I do not mean an actual piece of proxy software
> running on a machine somewhere. I mean just a data point in the
> ARIA-generated topology that can be used as a barrier of sorts when
> constructing the task graph -- because the task graph follows
> relationships, the edges of the topology. It could be that we discover in
> our POC that this is not needed, because actually the substitution node is
> already part of our topology data model and we might be able to easily take
> that into account when generating the task graph.)
>
> 3) Live service composition. I think this is a fantasy of some people: that
> ARIA would be able to take existing, "running" service chains and run
> workflows with them. I do think this is a solvable problem, but not via
> substitution mapping per se. A solution could involve deployment of a proxy
> service (which could actually be encapsulated in a substitution mapping,
> but doesn't have to be), or configuring specialized virtual ports via an
> SDN controller, or via logical proxy nodes created via an inspection tool,
> etc. I cannot believe that there is a one-size-fits-all solution to this
> problem. The dream of NFV might be to have everything connect to each other
> like LEGO blocks, but there are far too many protocols, configuration, and
> security standards. I think point #2, what I called "static service
> composition," is a realistic compromise and within the scope of what TOSCA
> promises.
>
>
> On Fri, Aug 11, 2017 at 12:08 PM, DeWayne Filppi 
> wrote:
>
> > To my eyes, the spec doesn't speak of runtime service substituion, but
> > parsetime template composition.  IOW, substitution mapping is a fancy
> > "include", or the equivalent of an interface definition.  Is it
> understood
> > by the ARIA team that this includes proxying of running services?  IOW,
> if
> > my template requires a database service that my template does *not* want
> to
> > control the lifecycle of, I can "substitution map" an instance of a
> > template (i.e. a running service)?  This would be a lovely feature, but
> > it's not really a "substitution map", rather more of a "service proxy"
> (as
> > 

Re: Service Composition / Substitution Mapping

[Tal Liron]

OK, let me try to organize this differently. Three potential conceptions
here:

1) A "fancy include," as you say. All that would happen here is that the
TOSCA parser would automatically find the service template to include from
some kind of repository of recognized service templates, and just include
that. The language in the TOSCA spec suggests that this is not the
intention: it is something that happens at the "orchestrator," not the
"parser."

2) Static service composition. This happens not at the parsing stage, but
rather the instantiation stage, where reqs-and-caps happens. I think this
is what is intended: substitution mapping is specifically about mapping
reqs-and-caps. And this is also where things like scalability and placement
happen: think of a requirement matching a capability, but that capability
not having enough capacity. So, the node might be able to scale out: in the
case of substitution, this would mean duplicating an entire service
instance. My understanding is that this is what is intended by TOSCA, and
it's entirely within the scope of what we can do. We've recently just
refactored the instantiation phase into a new "topology" module where
exactly all this logic is concentrated. So think of it this way -- ARIA has
three parts: "parser," "topology manager," and "workflow engine"
("orchestrator").

(I think I might have confused some thing here a bit when mentioning a
"logical proxy node." I do not mean an actual piece of proxy software
running on a machine somewhere. I mean just a data point in the
ARIA-generated topology that can be used as a barrier of sorts when
constructing the task graph -- because the task graph follows
relationships, the edges of the topology. It could be that we discover in
our POC that this is not needed, because actually the substitution node is
already part of our topology data model and we might be able to easily take
that into account when generating the task graph.)

3) Live service composition. I think this is a fantasy of some people: that
ARIA would be able to take existing, "running" service chains and run
workflows with them. I do think this is a solvable problem, but not via
substitution mapping per se. A solution could involve deployment of a proxy
service (which could actually be encapsulated in a substitution mapping,
but doesn't have to be), or configuring specialized virtual ports via an
SDN controller, or via logical proxy nodes created via an inspection tool,
etc. I cannot believe that there is a one-size-fits-all solution to this
problem. The dream of NFV might be to have everything connect to each other
like LEGO blocks, but there are far too many protocols, configuration, and
security standards. I think point #2, what I called "static service
composition," is a realistic compromise and within the scope of what TOSCA
promises.


On Fri, Aug 11, 2017 at 12:08 PM, DeWayne Filppi 
wrote:

> To my eyes, the spec doesn't speak of runtime service substituion, but
> parsetime template composition.  IOW, substitution mapping is a fancy
> "include", or the equivalent of an interface definition.  Is it understood
> by the ARIA team that this includes proxying of running services?  IOW, if
> my template requires a database service that my template does *not* want to
> control the lifecycle of, I can "substitution map" an instance of a
> template (i.e. a running service)?  This would be a lovely feature, but
> it's not really a "substitution map", rather more of a "service proxy" (as
> implemented as a plugin in Cloudify).   Just trying to clarify.  Maybe the
> community thinks that "substitution map" as something that occurs beyond
> parsetime, or should.
>
> On Fri, Aug 11, 2017 at 9:52 AM, Tal Liron  wrote:
>
> > Well, DJ, it's all just opinion at this stage and feedback is welcome!
> >
> > Option 1:
> > > To look at satisfying nodes present in a substituting service,
> > > Have these nodes part of the newly created service and remove the
> > > substituting service(nodes with different ID's. Also we are very much
> in
> > > favor of  UUID )
> > > With this approach I guess the substituting service should not
> > > have any associated workflows running. If at all an workflow execution
> is
> > > already triggered I hope this service will not be considered for
> > > substitution.
> > > I hope this is the correct approach when we are looking at a
> > > service for the substitution
> > >
> >
> > Yes, this is a good idea. It would be easy to discover this according to
> > the stored node state -- just make sure that all nodes are in a stable
> > state before composition.
> >
> > This leads to a general issue: before composition, the substituting
> > services must be validated in some way before composition begins.
> >
> > Also, let's all start using TOSCA terminology here: the containing
> service
> > is called the "top-level service," and the contained services are called
> > 

Re: Service Composition / Substitution Mapping

[DeWayne Filppi]

To my eyes, the spec doesn't speak of runtime service substituion, but
parsetime template composition.  IOW, substitution mapping is a fancy
"include", or the equivalent of an interface definition.  Is it understood
by the ARIA team that this includes proxying of running services?  IOW, if
my template requires a database service that my template does *not* want to
control the lifecycle of, I can "substitution map" an instance of a
template (i.e. a running service)?  This would be a lovely feature, but
it's not really a "substitution map", rather more of a "service proxy" (as
implemented as a plugin in Cloudify).   Just trying to clarify.  Maybe the
community thinks that "substitution map" as something that occurs beyond
parsetime, or should.

On Fri, Aug 11, 2017 at 9:52 AM, Tal Liron  wrote:

> Well, DJ, it's all just opinion at this stage and feedback is welcome!
>
> Option 1:
> > To look at satisfying nodes present in a substituting service,
> > Have these nodes part of the newly created service and remove the
> > substituting service(nodes with different ID's. Also we are very much in
> > favor of  UUID )
> > With this approach I guess the substituting service should not
> > have any associated workflows running. If at all an workflow execution is
> > already triggered I hope this service will not be considered for
> > substitution.
> > I hope this is the correct approach when we are looking at a
> > service for the substitution
> >
>
> Yes, this is a good idea. It would be easy to discover this according to
> the stored node state -- just make sure that all nodes are in a stable
> state before composition.
>
> This leads to a general issue: before composition, the substituting
> services must be validated in some way before composition begins.
>
> Also, let's all start using TOSCA terminology here: the containing service
> is called the "top-level service," and the contained services are called
> "substituting services."
>
> Also, we keep using trivial examples, but it's definitely possible for a
> top-level service to require several substituting services at the same
> time. I can definitely see such things happening in NFV with even simple
> service chains. Basically every VNF could be a substituting service.
>
> So, actually one of the validations would be to make sure you do not create
> circular composition: if the top-level also has substitution mappings, you
> need to make sure that one of the substituting ones doesn't require it. :)
> Not a very likely situation, but it would lead to failure.
>
>
>
> > Option 2:
> > While creating a service look at the req-caps at the
> > service-template level and create a service including the nodes provided
> by
> > the substituting service-template. With this approach there would not be
> > any   service created from the service-template which is providing the
> > substitution functionality. The service-template would remain the same
> but
> > only the service would be added with extra nodes.
> >
> > Are you considering both option 1 & 2 for the implementation ? If not
> > which one do you feel which take priority. I see option 2 at this stage
> > could be the best possible approach
> > Also could you please let me know a tentative time for this feature to be
> > available?
> >
>
> I think both options 1 and 2 make sense and are useful, and actually one is
> a subset of the other.
>
> With option #2 (substituting a service template), it means new nodes are
> instantiated and the composed service would include all nodes. So, an
> "install" workflow would install everything at once. In this case we do
> need to fix the lifecycle workflows to be "boundary aware," so that
> workflows of substituting service nodes are part of their own task graph. I
> think that possibly using a logical proxy node in between might solve this
> situation automatically.
>
> With option #1 (substituting a service) the substituting nodes might
> already be installed. Or not (they might have been instantiated but still
> not installed). So, the lifecycle workflows should only work on the nodes
> that have not yet been installed.
>
> The point is that we just need to beef up the lifecycle workflows to
> properly work with boundaries and with a mix of nodes in different states.
> If they can do that, then they can handle any kind of service composition,
> whether it's option #1 or option #2.
>
> I don't think we can provide a timeline yet, but I will say that we are in
> the research phase and may have a POC soon. Avia is in charge of this JIRA,
> so I will let him chime in with the current state of things.
>

Re: Service Composition / Substitution Mapping

[Tal Liron]

Well, DJ, it's all just opinion at this stage and feedback is welcome!

Option 1:
> To look at satisfying nodes present in a substituting service,
> Have these nodes part of the newly created service and remove the
> substituting service(nodes with different ID's. Also we are very much in
> favor of  UUID )
> With this approach I guess the substituting service should not
> have any associated workflows running. If at all an workflow execution is
> already triggered I hope this service will not be considered for
> substitution.
> I hope this is the correct approach when we are looking at a
> service for the substitution
>

Yes, this is a good idea. It would be easy to discover this according to
the stored node state -- just make sure that all nodes are in a stable
state before composition.

This leads to a general issue: before composition, the substituting
services must be validated in some way before composition begins.

Also, let's all start using TOSCA terminology here: the containing service
is called the "top-level service," and the contained services are called
"substituting services."

Also, we keep using trivial examples, but it's definitely possible for a
top-level service to require several substituting services at the same
time. I can definitely see such things happening in NFV with even simple
service chains. Basically every VNF could be a substituting service.

So, actually one of the validations would be to make sure you do not create
circular composition: if the top-level also has substitution mappings, you
need to make sure that one of the substituting ones doesn't require it. :)
Not a very likely situation, but it would lead to failure.



> Option 2:
> While creating a service look at the req-caps at the
> service-template level and create a service including the nodes provided by
> the substituting service-template. With this approach there would not be
> any   service created from the service-template which is providing the
> substitution functionality. The service-template would remain the same but
> only the service would be added with extra nodes.
>
> Are you considering both option 1 & 2 for the implementation ? If not
> which one do you feel which take priority. I see option 2 at this stage
> could be the best possible approach
> Also could you please let me know a tentative time for this feature to be
> available?
>

I think both options 1 and 2 make sense and are useful, and actually one is
a subset of the other.

With option #2 (substituting a service template), it means new nodes are
instantiated and the composed service would include all nodes. So, an
"install" workflow would install everything at once. In this case we do
need to fix the lifecycle workflows to be "boundary aware," so that
workflows of substituting service nodes are part of their own task graph. I
think that possibly using a logical proxy node in between might solve this
situation automatically.

With option #1 (substituting a service) the substituting nodes might
already be installed. Or not (they might have been instantiated but still
not installed). So, the lifecycle workflows should only work on the nodes
that have not yet been installed.

The point is that we just need to beef up the lifecycle workflows to
properly work with boundaries and with a mix of nodes in different states.
If they can do that, then they can handle any kind of service composition,
whether it's option #1 or option #2.

I don't think we can provide a timeline yet, but I will say that we are in
the research phase and may have a POC soon. Avia is in charge of this JIRA,
so I will let him chime in with the current state of things.

RE: Service Composition / Substitution Mapping

[D Jayachandran]

Hi Tal,

Thanks for the explanation.

So we have 2 options when the reqs-caps are not met within the same 
service-template

Option 1:
To look at satisfying nodes present in a substituting service, Have 
these nodes part of the newly created service and remove the substituting 
service(nodes with different ID's. Also we are very much in favor of  UUID )
With this approach I guess the substituting service should not have any 
associated workflows running. If at all an workflow execution is already 
triggered I hope this service will not be considered for substitution.
I hope this is the correct approach when we are looking at a service 
for the substitution

Option 2:
While creating a service look at the req-caps at the service-template 
level and create a service including the nodes provided by the substituting 
service-template. With this approach there would not be any   service created 
from the service-template which is providing the substitution functionality. 
The service-template would remain the same but only the service would be added 
with extra nodes.

Are you considering both option 1 & 2 for the implementation ? If not which one 
do you feel which take priority. I see option 2 at this stage could be the best 
possible approach
Also could you please let me know a tentative time for this feature to be 
available?


Regards,
DJ
-Original Message-
From: Tal Liron [mailto:t...@cloudify.co] 
Sent: Thursday, August 10, 2017 10:09 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping

Thanks for the feedback, DJ. What I wrote was just ideas for now, we're still 
in the investigation phase and haven't implemented anything yet.

1. The reqs-and-caps engine by default will always look for satisfiable
> capabilities within the currently instantiated service. HOWEVER, if 
> such a capability is not present, the option is there to look for 
> another instantiated service that exposes the capabilities in substitution 
> mappings.
> [DJ] - When you say option is there to look for another 
> instantiated service is this an available option with current ARIA ?
>  - When you say instantiated service, is it the 
> service or the real world service ?
>  - I think the 3rd point of yours is related to this 
> service level mapping. When you say a special node would be added to 
> the current service, will that node be unique across service A and 
> service B(instantiated service) ? Will a life-cycle operation would be 
> called for that node which is added to service A as part of the workflow 
> execution ?
>

I don't think it's reasonable for ARIA to work with a "real world service"
if it hasn't been modeled yet in some way. I do have a dream of someday having 
such a tool: take an existing cloud service and produce a basic TOSCA service 
*and* service template for it. But for now I think it's reasonable to expect 
the user to at least model the whole "real world"
service as some kind of logical node.

The question you ask about lifecycle operations are the right ones. In my 
opinion, the new "composed service" should be a service instance in every 
respect, so workflows would indeed happen on all nodes, including the 
sub-services that were added. Otherwise, why do composition at all? The whole 
point is to combine everything together.

An interesting question is what happens to nodes after that get "composed"
into another service. From what I say above, it means their service IDs should 
change, and in fact the original service would disappear. (By the way, things 
like this are another reason I prefer UUIDs for nodes rather than have them be 
sequential within the service: nodes should be allowed to move around between 
services.)

The only way you would know that these added nodes came from a another service 
template is by following their node_template_fk to their service_template_fk. 
Otherwise they are all in the same service.


> 2. If we DON'T have another instantiated service, but DO have a 
> service template that could fit the bill, perhaps we need to 
> instantiate that other service first. One obvious option is to do this 
> automatically. But I feel like this can create unforeseen consequences 
> -- for example, some dummy test template that someone happened to have 
> in the database might get instantiated by mistake. Also, it might need 
> to trigger multiple install workflows at once... a big mess. So I 
> suggest that instead we provide a very detailed validation error here 
> saying that the requirement cannot be satisfied, HOWEVER there exist 
> service templates A, B, and C that can substitute for us, so maybe the 
> nice user would like to instantiate them first? This seems very reasonable to 
> me.
> [DJ] - Just to understand m

Re: Service Composition / Substitution Mapping

[Tal Liron]

Thanks for the feedback, DJ. What I wrote was just ideas for now, we're
still in the investigation phase and haven't implemented anything yet.

1. The reqs-and-caps engine by default will always look for satisfiable
> capabilities within the currently instantiated service. HOWEVER, if such a
> capability is not present, the option is there to look for another
> instantiated service that exposes the capabilities in substitution mappings.
> [DJ] - When you say option is there to look for another
> instantiated service is this an available option with current ARIA ?
>  - When you say instantiated service, is it the service or
> the real world service ?
>  - I think the 3rd point of yours is related to this
> service level mapping. When you say a special node would be added to the
> current service, will that node be unique across service A and service
> B(instantiated service) ? Will a life-cycle operation would be called for
> that node which is added to service A as part of the workflow execution ?
>

I don't think it's reasonable for ARIA to work with a "real world service"
if it hasn't been modeled yet in some way. I do have a dream of someday
having such a tool: take an existing cloud service and produce a basic
TOSCA service *and* service template for it. But for now I think it's
reasonable to expect the user to at least model the whole "real world"
service as some kind of logical node.

The question you ask about lifecycle operations are the right ones. In my
opinion, the new "composed service" should be a service instance in every
respect, so workflows would indeed happen on all nodes, including the
sub-services that were added. Otherwise, why do composition at all? The
whole point is to combine everything together.

An interesting question is what happens to nodes after that get "composed"
into another service. From what I say above, it means their service IDs
should change, and in fact the original service would disappear. (By the
way, things like this are another reason I prefer UUIDs for nodes rather
than have them be sequential within the service: nodes should be allowed to
move around between services.)

The only way you would know that these added nodes came from a another
service template is by following their node_template_fk to their
service_template_fk. Otherwise they are all in the same service.


> 2. If we DON'T have another instantiated service, but DO have a service
> template that could fit the bill, perhaps we need to instantiate that other
> service first. One obvious option is to do this automatically. But I feel
> like this can create unforeseen consequences -- for example, some dummy
> test template that someone happened to have in the database might get
> instantiated by mistake. Also, it might need to trigger multiple install
> workflows at once... a big mess. So I suggest that instead we provide a
> very detailed validation error here saying that the requirement cannot be
> satisfied, HOWEVER there exist service templates A, B, and C that can
> substitute for us, so maybe the nice user would like to instantiate them
> first? This seems very reasonable to me.
> [DJ] - Just to understand more on this, Let us assume we have
> service-template A and service-template B. Am trying to create a service A
> from service-template A. One of the node is abstract and this capability is
> provided by node from service-template B.
> - Now I assume service A will have node contributed by
> service-template B and also its nodes. Will this approach I don't see a
> need for multiple workflows.
> - Or is it like service B would also be created
> automatically. In that case how would the workflow be called for service B ?
> - As you stated we have this challenge with multiple
> service-template providing the same capabilities on which one to use.
> - Finally am not getting the exact meaning of the last
> statement of yours "HOWEVER there exist service templates A, B, and C that
> can substitute for us, so maybe the nice user would like to instantiate
> them first? This seems very reasonable to me". I assume you are talking
> having a provision where the user can mention the service-template to be
> used
>

I was thinking out loud there, comparing the two options. Thinking more
about it now, I think it's fine to run a normal "install" workflow on the
big composed service (as I said, it should be treated as a regular service
instance), so I see no problem with instantiating one big service
automatically for the user. No need for the user to instantiate the
sub-service first.

HOWEVER, during instantiation the reqs-and-caps engine should be careful to
consider each sub-service a separate "zone" for satisfying capabilities.
You don't want a requirement in one service template grabbing a capability
of a node in the other. The service is composed, but the service templates
are still separate.

RE: Service Composition / Substitution Mapping

[D Jayachandran]

Hi Tal,

I remember we had the conversion of substitution mapping sometime ago, where we 
agreed upon of looking at available service-models(service-templates) when we 
dont find a satisfiable capability within our service. I just want to make sure 
if we are on the same page as the implementation work is started for this. Also 
I have few questions with respect to the points you had put forth. Please find 
my comments in-line against each of your points.

1. The reqs-and-caps engine by default will always look for satisfiable 
capabilities within the currently instantiated service. HOWEVER, if such a 
capability is not present, the option is there to look for another instantiated 
service that exposes the capabilities in substitution mappings.
[DJ] - When you say option is there to look for another instantiated 
service is this an available option with current ARIA ?
 - When you say instantiated service, is it the service or the 
real world service ?
 - I think the 3rd point of yours is related to this service 
level mapping. When you say a special node would be added to the current 
service, will that node be unique across service A and service B(instantiated 
service) ? Will a life-cycle operation would be called for that node which is 
added to service A as part of the workflow execution ? 


2. If we DON'T have another instantiated service, but DO have a service 
template that could fit the bill, perhaps we need to instantiate that other 
service first. One obvious option is to do this automatically. But I feel like 
this can create unforeseen consequences -- for example, some dummy test 
template that someone happened to have in the database might get instantiated 
by mistake. Also, it might need to trigger multiple install workflows at 
once... a big mess. So I suggest that instead we provide a very detailed 
validation error here saying that the requirement cannot be satisfied, HOWEVER 
there exist service templates A, B, and C that can substitute for us, so maybe 
the nice user would like to instantiate them first? This seems very reasonable 
to me.
[DJ] - Just to understand more on this, Let us assume we have 
service-template A and service-template B. Am trying to create a service A from 
service-template A. One of the node is abstract and this capability is provided 
by node from service-template B. 
- Now I assume service A will have node contributed by 
service-template B and also its nodes. Will this approach I don't see a need 
for multiple workflows.
- Or is it like service B would also be created automatically. 
In that case how would the workflow be called for service B ?
- As you stated we have this challenge with multiple 
service-template providing the same capabilities on which one to use.
- Finally am not getting the exact meaning of the last 
statement of yours "HOWEVER there exist service templates A, B, and C that can 
substitute for us, so maybe the nice user would like to instantiate them first? 
This seems very reasonable to me". I assume you are talking having a provision 
where the user can mention the service-template to be used 

3. If indeed another service satisfies this, a special node is added to the 
current service (with the correct type -- but without a service template 
foreign key), which serves as a proxy of the other service template. I'm not 
sure how we would mark this exactly. We can't use the service_fk field, because 
it's still in our current service. So perhaps there's need of a new fk field, 
maybe substituted_service_fk?

The above might be "sensible defaults," but it seems to me that users really 
need control over this. So I propose to add a new aria.Composition policy that 
would let you provide hints for this mechanism. For example, you might want to 
"filter" the target service by service template name and even by metadata in 
the service template. For example, you might want to require version 1.2.2 of a 
specific service, no less.

Regards,
DJ
-Original Message-
From: Avia Efrat [mailto:a...@cloudify.co] 
Sent: Wednesday, August 09, 2017 6:28 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping

Currently, the support for substitution mapping is extremely limited. the 
substitution_mapping section is parsed and validated, but nothing more.

However, supporting this feature (and hence, allowing for service
composition) is of high priority. Currently, we are at the the designing phase 
- going over the spec to identify vague and ambiguous parts, and consolidating 
TOSCA's pov regarding substitution mapping with the current ARIA implementation.

Relevant Jira issue:
https://issues.apache.org/jira/browse/ARIA-292

On Tue, Aug 8, 2017 at 11:03 PM, Steve Baillargeon < 
steve.baillarg...@ericsson.com> wrote:

> Hi
> I have looked at the data yo

Re: Service Composition / Substitution Mapping

[Avia Efrat]

Currently, the support for substitution mapping is extremely limited. the
substitution_mapping section is parsed and validated, but nothing more.

However, supporting this feature (and hence, allowing for service
composition) is of high priority. Currently, we are at the the designing
phase - going over the spec to identify vague and ambiguous parts, and
consolidating TOSCA's pov regarding substitution mapping with the current
ARIA implementation.

Relevant Jira issue:
https://issues.apache.org/jira/browse/ARIA-292

On Tue, Aug 8, 2017 at 11:03 PM, Steve Baillargeon <
steve.baillarg...@ericsson.com> wrote:

> Hi
> I have looked at the data you have provided.
> I am trying to catch up (understand) to current ARIA support for
> substitution mapping.
> What is supported today and what are the limitations/considerations?
> What is missing (hence the need for full implementation)?
> Do you have an working example or set of guidelines for using substitution
> mapping today?
> Or should I avoid it completely for now?
>
> - Steve B
>
> -Original Message-
> From: Tal Liron [mailto:t...@cloudify.co]
> Sent: Monday, August 07, 2017 8:41 PM
> To: dev@ariatosca.incubator.apache.org
> Subject: Re: Service Composition / Substitution Mapping
>
> Well, this is exactly what policies are for. :)
>
> Again, I think the rule of thumb should be that users put policies in place
> *only* if the defaults do not suffice.
>
> On Mon, Aug 7, 2017 at 6:42 PM, Ran Ziv <r...@cloudify.co> wrote:
>
> > The sensible defaults Tal's mentioned sound indeed sensible to me.
> > I'd also like users to have control over this, though I'm a bit
> > worried about us getting too carried away with how arbitrarily we use
> > policies for configuring, well, pretty much anything. It might not be
> > a problem right now but I'm not certain that will remain the case in
> > the future when the number of them grows..
> >
> >
> > On Wed, Aug 2, 2017 at 7:14 PM, Tal Liron <t...@cloudify.co> wrote:
> >
> > > Our goal with adding new "conventions" to ARIA, such as policies, is
> > > to always make them optional. The idea is that a plain-vanilla TOSCA
> > template
> > > would "just work" in ARIA via sensible defaults. The extra stuff is
> > > there if you know you are using ARIA and you want to make use of its
> features.
> > > (The opposite is true, too: we make sure that any additions are
> > > still
> > pure
> > > TOSCA and would be parsed validly by other TOSCA parsers.)
> > >
> > > On Wed, Aug 2, 2017 at 9:08 AM, DeWayne Filppi <dewa...@cloudify.co>
> > > wrote:
> > >
> > > > Cool.  Missed that.  That leaves things almost completely wide
> > > > open
> > from
> > > > the orchestrator side, IOW few predefined keys.  Too few IMHO, but
> > > > if everyone uses ARIA conventions it could work.
> > > >
> > > > On Tue, Aug 1, 2017 at 11:49 PM, Tal Liron <t...@cloudify.co> wrote:
> > > >
> > > > > I agree! Luckily metadata exists in the 1.0 spec. :)
> > > > >
> > > > > http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-
> > > > > YAML/v1.0/cos01/TOSCA-Simple-Profile-YAML-v1.0-cos01.html#_
> > > Toc379455044
> > > > >
> > > > > On Tue, Aug 1, 2017 at 7:16 PM, DeWayne Filppi
> > > > > <dewa...@cloudify.co>
> > > > > wrote:
> > > > >
> > > > > > It occurs that it might be useful to be able to tag service
> > templates
> > > > > with
> > > > > > arbitrary meta-data.  Perhaps at one level carried forward
> > > > > > from a
> > > CSAR
> > > > > > manifest, but also user definable.  This would allow
> > > > > > inter-service references to be definitive, if desired.  This
> > > > > > could be implicitly
> > > > > defined
> > > > > > as a capability by the orchestrator, but some kind of special
> > > > requirement
> > > > > > type(s) would be needed to utilize it.  This way, external
> > > > > > repos
> > > could
> > > > be
> > > > > > used safely and directly without the separate load step.
> > > > > >
> > > > > > On Tue, Aug 1, 2017 at 12:43 PM, Tal Liron <t...@cloudify.co>
> > wrote:
> > > > > >
> > > > > > > Thanks for the kudos. :)
> > > > > > >
> > > > >

RE: Service Composition / Substitution Mapping

[Steve Baillargeon]

Hi
I have looked at the data you have provided.
I am trying to catch up (understand) to current ARIA support for substitution 
mapping.
What is supported today and what are the limitations/considerations?
What is missing (hence the need for full implementation)?
Do you have an working example or set of guidelines for using substitution 
mapping today? 
Or should I avoid it completely for now?

- Steve B

-Original Message-
From: Tal Liron [mailto:t...@cloudify.co] 
Sent: Monday, August 07, 2017 8:41 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping

Well, this is exactly what policies are for. :)

Again, I think the rule of thumb should be that users put policies in place
*only* if the defaults do not suffice.

On Mon, Aug 7, 2017 at 6:42 PM, Ran Ziv <r...@cloudify.co> wrote:

> The sensible defaults Tal's mentioned sound indeed sensible to me.
> I'd also like users to have control over this, though I'm a bit 
> worried about us getting too carried away with how arbitrarily we use 
> policies for configuring, well, pretty much anything. It might not be 
> a problem right now but I'm not certain that will remain the case in 
> the future when the number of them grows..
>
>
> On Wed, Aug 2, 2017 at 7:14 PM, Tal Liron <t...@cloudify.co> wrote:
>
> > Our goal with adding new "conventions" to ARIA, such as policies, is 
> > to always make them optional. The idea is that a plain-vanilla TOSCA
> template
> > would "just work" in ARIA via sensible defaults. The extra stuff is 
> > there if you know you are using ARIA and you want to make use of its 
> > features.
> > (The opposite is true, too: we make sure that any additions are 
> > still
> pure
> > TOSCA and would be parsed validly by other TOSCA parsers.)
> >
> > On Wed, Aug 2, 2017 at 9:08 AM, DeWayne Filppi <dewa...@cloudify.co>
> > wrote:
> >
> > > Cool.  Missed that.  That leaves things almost completely wide 
> > > open
> from
> > > the orchestrator side, IOW few predefined keys.  Too few IMHO, but 
> > > if everyone uses ARIA conventions it could work.
> > >
> > > On Tue, Aug 1, 2017 at 11:49 PM, Tal Liron <t...@cloudify.co> wrote:
> > >
> > > > I agree! Luckily metadata exists in the 1.0 spec. :)
> > > >
> > > > http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-
> > > > YAML/v1.0/cos01/TOSCA-Simple-Profile-YAML-v1.0-cos01.html#_
> > Toc379455044
> > > >
> > > > On Tue, Aug 1, 2017 at 7:16 PM, DeWayne Filppi 
> > > > <dewa...@cloudify.co>
> > > > wrote:
> > > >
> > > > > It occurs that it might be useful to be able to tag service
> templates
> > > > with
> > > > > arbitrary meta-data.  Perhaps at one level carried forward 
> > > > > from a
> > CSAR
> > > > > manifest, but also user definable.  This would allow 
> > > > > inter-service references to be definitive, if desired.  This 
> > > > > could be implicitly
> > > > defined
> > > > > as a capability by the orchestrator, but some kind of special
> > > requirement
> > > > > type(s) would be needed to utilize it.  This way, external 
> > > > > repos
> > could
> > > be
> > > > > used safely and directly without the separate load step.
> > > > >
> > > > > On Tue, Aug 1, 2017 at 12:43 PM, Tal Liron <t...@cloudify.co>
> wrote:
> > > > >
> > > > > > Thanks for the kudos. :)
> > > > > >
> > > > > > This topic was discussed on this list a while ago. It's 
> > > > > > indeed
> > tricky
> > > > to
> > > > > > get right, because TOSCA leaves a lot of room for the
> orchestrator
> > to
> > > > > > implement.
> > > > > >
> > > > > > I'm thinking of it working something like this:
> > > > > >
> > > > > > 1. The reqs-and-caps engine by default will always look for
> > > satisfiable
> > > > > > capabilities within the currently instantiated service. 
> > > > > > HOWEVER,
> if
> > > > such
> > > > > a
> > > > > > capability is not present, the option is there to look for
> another
> > > > > > instantiated service that exposes the capabilities in
> substitution
> > > > > > mappings.
> > > > > >
> > > > &

Re: Service Composition / Substitution Mapping

[Tal Liron]

Well, this is exactly what policies are for. :)

Again, I think the rule of thumb should be that users put policies in place
*only* if the defaults do not suffice.

On Mon, Aug 7, 2017 at 6:42 PM, Ran Ziv  wrote:

> The sensible defaults Tal's mentioned sound indeed sensible to me.
> I'd also like users to have control over this, though I'm a bit worried
> about us getting too carried away with how arbitrarily we use policies for
> configuring, well, pretty much anything. It might not be a problem right
> now but I'm not certain that will remain the case in the future when the
> number of them grows..
>
>
> On Wed, Aug 2, 2017 at 7:14 PM, Tal Liron  wrote:
>
> > Our goal with adding new "conventions" to ARIA, such as policies, is to
> > always make them optional. The idea is that a plain-vanilla TOSCA
> template
> > would "just work" in ARIA via sensible defaults. The extra stuff is there
> > if you know you are using ARIA and you want to make use of its features.
> > (The opposite is true, too: we make sure that any additions are still
> pure
> > TOSCA and would be parsed validly by other TOSCA parsers.)
> >
> > On Wed, Aug 2, 2017 at 9:08 AM, DeWayne Filppi 
> > wrote:
> >
> > > Cool.  Missed that.  That leaves things almost completely wide open
> from
> > > the orchestrator side, IOW few predefined keys.  Too few IMHO, but if
> > > everyone uses ARIA conventions it could work.
> > >
> > > On Tue, Aug 1, 2017 at 11:49 PM, Tal Liron  wrote:
> > >
> > > > I agree! Luckily metadata exists in the 1.0 spec. :)
> > > >
> > > > http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-
> > > > YAML/v1.0/cos01/TOSCA-Simple-Profile-YAML-v1.0-cos01.html#_
> > Toc379455044
> > > >
> > > > On Tue, Aug 1, 2017 at 7:16 PM, DeWayne Filppi 
> > > > wrote:
> > > >
> > > > > It occurs that it might be useful to be able to tag service
> templates
> > > > with
> > > > > arbitrary meta-data.  Perhaps at one level carried forward from a
> > CSAR
> > > > > manifest, but also user definable.  This would allow inter-service
> > > > > references to be definitive, if desired.  This could be implicitly
> > > > defined
> > > > > as a capability by the orchestrator, but some kind of special
> > > requirement
> > > > > type(s) would be needed to utilize it.  This way, external repos
> > could
> > > be
> > > > > used safely and directly without the separate load step.
> > > > >
> > > > > On Tue, Aug 1, 2017 at 12:43 PM, Tal Liron 
> wrote:
> > > > >
> > > > > > Thanks for the kudos. :)
> > > > > >
> > > > > > This topic was discussed on this list a while ago. It's indeed
> > tricky
> > > > to
> > > > > > get right, because TOSCA leaves a lot of room for the
> orchestrator
> > to
> > > > > > implement.
> > > > > >
> > > > > > I'm thinking of it working something like this:
> > > > > >
> > > > > > 1. The reqs-and-caps engine by default will always look for
> > > satisfiable
> > > > > > capabilities within the currently instantiated service. HOWEVER,
> if
> > > > such
> > > > > a
> > > > > > capability is not present, the option is there to look for
> another
> > > > > > instantiated service that exposes the capabilities in
> substitution
> > > > > > mappings.
> > > > > >
> > > > > > 2. If we DON'T have another instantiated service, but DO have a
> > > service
> > > > > > template that could fit the bill, perhaps we need to instantiate
> > that
> > > > > other
> > > > > > service first. One obvious option is to do this automatically.
> But
> > I
> > > > feel
> > > > > > like this can create unforeseen consequences -- for example, some
> > > dummy
> > > > > > test template that someone happened to have in the database might
> > get
> > > > > > instantiated by mistake. Also, it might need to trigger multiple
> > > > install
> > > > > > workflows at once... a big mess. So I suggest that instead we
> > > provide a
> > > > > > very detailed validation error here saying that the requirement
> > > cannot
> > > > be
> > > > > > satisfied, HOWEVER there exist service templates A, B, and C that
> > can
> > > > > > substitute for us, so maybe the nice user would like to
> instantiate
> > > > them
> > > > > > first? This seems very reasonable to me.
> > > > > >
> > > > > > 3. If indeed another service satisfies this, a special node is
> > added
> > > to
> > > > > the
> > > > > > current service (with the correct type -- but without a service
> > > > template
> > > > > > foreign key), which serves as a proxy of the other service
> > template.
> > > > I'm
> > > > > > not sure how we would mark this exactly. We can't use the
> > service_fk
> > > > > field,
> > > > > > because it's still in our current service. So perhaps there's
> need
> > > of a
> > > > > new
> > > > > > fk field, maybe substituted_service_fk?
> > > > > >
> > > > > > The above might be "sensible defaults," but it seems to me that
> > users
> > > > > > really need control over 

Re: Service Composition / Substitution Mapping

[Ran Ziv]

The sensible defaults Tal's mentioned sound indeed sensible to me.
I'd also like users to have control over this, though I'm a bit worried
about us getting too carried away with how arbitrarily we use policies for
configuring, well, pretty much anything. It might not be a problem right
now but I'm not certain that will remain the case in the future when the
number of them grows..


On Wed, Aug 2, 2017 at 7:14 PM, Tal Liron  wrote:

> Our goal with adding new "conventions" to ARIA, such as policies, is to
> always make them optional. The idea is that a plain-vanilla TOSCA template
> would "just work" in ARIA via sensible defaults. The extra stuff is there
> if you know you are using ARIA and you want to make use of its features.
> (The opposite is true, too: we make sure that any additions are still pure
> TOSCA and would be parsed validly by other TOSCA parsers.)
>
> On Wed, Aug 2, 2017 at 9:08 AM, DeWayne Filppi 
> wrote:
>
> > Cool.  Missed that.  That leaves things almost completely wide open from
> > the orchestrator side, IOW few predefined keys.  Too few IMHO, but if
> > everyone uses ARIA conventions it could work.
> >
> > On Tue, Aug 1, 2017 at 11:49 PM, Tal Liron  wrote:
> >
> > > I agree! Luckily metadata exists in the 1.0 spec. :)
> > >
> > > http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-
> > > YAML/v1.0/cos01/TOSCA-Simple-Profile-YAML-v1.0-cos01.html#_
> Toc379455044
> > >
> > > On Tue, Aug 1, 2017 at 7:16 PM, DeWayne Filppi 
> > > wrote:
> > >
> > > > It occurs that it might be useful to be able to tag service templates
> > > with
> > > > arbitrary meta-data.  Perhaps at one level carried forward from a
> CSAR
> > > > manifest, but also user definable.  This would allow inter-service
> > > > references to be definitive, if desired.  This could be implicitly
> > > defined
> > > > as a capability by the orchestrator, but some kind of special
> > requirement
> > > > type(s) would be needed to utilize it.  This way, external repos
> could
> > be
> > > > used safely and directly without the separate load step.
> > > >
> > > > On Tue, Aug 1, 2017 at 12:43 PM, Tal Liron  wrote:
> > > >
> > > > > Thanks for the kudos. :)
> > > > >
> > > > > This topic was discussed on this list a while ago. It's indeed
> tricky
> > > to
> > > > > get right, because TOSCA leaves a lot of room for the orchestrator
> to
> > > > > implement.
> > > > >
> > > > > I'm thinking of it working something like this:
> > > > >
> > > > > 1. The reqs-and-caps engine by default will always look for
> > satisfiable
> > > > > capabilities within the currently instantiated service. HOWEVER, if
> > > such
> > > > a
> > > > > capability is not present, the option is there to look for another
> > > > > instantiated service that exposes the capabilities in substitution
> > > > > mappings.
> > > > >
> > > > > 2. If we DON'T have another instantiated service, but DO have a
> > service
> > > > > template that could fit the bill, perhaps we need to instantiate
> that
> > > > other
> > > > > service first. One obvious option is to do this automatically. But
> I
> > > feel
> > > > > like this can create unforeseen consequences -- for example, some
> > dummy
> > > > > test template that someone happened to have in the database might
> get
> > > > > instantiated by mistake. Also, it might need to trigger multiple
> > > install
> > > > > workflows at once... a big mess. So I suggest that instead we
> > provide a
> > > > > very detailed validation error here saying that the requirement
> > cannot
> > > be
> > > > > satisfied, HOWEVER there exist service templates A, B, and C that
> can
> > > > > substitute for us, so maybe the nice user would like to instantiate
> > > them
> > > > > first? This seems very reasonable to me.
> > > > >
> > > > > 3. If indeed another service satisfies this, a special node is
> added
> > to
> > > > the
> > > > > current service (with the correct type -- but without a service
> > > template
> > > > > foreign key), which serves as a proxy of the other service
> template.
> > > I'm
> > > > > not sure how we would mark this exactly. We can't use the
> service_fk
> > > > field,
> > > > > because it's still in our current service. So perhaps there's need
> > of a
> > > > new
> > > > > fk field, maybe substituted_service_fk?
> > > > >
> > > > > The above might be "sensible defaults," but it seems to me that
> users
> > > > > really need control over this. So I propose to add a new
> > > aria.Composition
> > > > > policy that would let you provide hints for this mechanism. For
> > > example,
> > > > > you might want to "filter" the target service by service template
> > name
> > > > and
> > > > > even by metadata in the service template. For example, you might
> want
> > > to
> > > > > require version 1.2.2 of a specific service, no less.
> > > > >
> > > > > Those are some quick thoughts. Exactly how such a policy would look
> 

Re: Service Composition / Substitution Mapping

[Tal Liron]

Our goal with adding new "conventions" to ARIA, such as policies, is to
always make them optional. The idea is that a plain-vanilla TOSCA template
would "just work" in ARIA via sensible defaults. The extra stuff is there
if you know you are using ARIA and you want to make use of its features.
(The opposite is true, too: we make sure that any additions are still pure
TOSCA and would be parsed validly by other TOSCA parsers.)

On Wed, Aug 2, 2017 at 9:08 AM, DeWayne Filppi  wrote:

> Cool.  Missed that.  That leaves things almost completely wide open from
> the orchestrator side, IOW few predefined keys.  Too few IMHO, but if
> everyone uses ARIA conventions it could work.
>
> On Tue, Aug 1, 2017 at 11:49 PM, Tal Liron  wrote:
>
> > I agree! Luckily metadata exists in the 1.0 spec. :)
> >
> > http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-
> > YAML/v1.0/cos01/TOSCA-Simple-Profile-YAML-v1.0-cos01.html#_Toc379455044
> >
> > On Tue, Aug 1, 2017 at 7:16 PM, DeWayne Filppi 
> > wrote:
> >
> > > It occurs that it might be useful to be able to tag service templates
> > with
> > > arbitrary meta-data.  Perhaps at one level carried forward from a CSAR
> > > manifest, but also user definable.  This would allow inter-service
> > > references to be definitive, if desired.  This could be implicitly
> > defined
> > > as a capability by the orchestrator, but some kind of special
> requirement
> > > type(s) would be needed to utilize it.  This way, external repos could
> be
> > > used safely and directly without the separate load step.
> > >
> > > On Tue, Aug 1, 2017 at 12:43 PM, Tal Liron  wrote:
> > >
> > > > Thanks for the kudos. :)
> > > >
> > > > This topic was discussed on this list a while ago. It's indeed tricky
> > to
> > > > get right, because TOSCA leaves a lot of room for the orchestrator to
> > > > implement.
> > > >
> > > > I'm thinking of it working something like this:
> > > >
> > > > 1. The reqs-and-caps engine by default will always look for
> satisfiable
> > > > capabilities within the currently instantiated service. HOWEVER, if
> > such
> > > a
> > > > capability is not present, the option is there to look for another
> > > > instantiated service that exposes the capabilities in substitution
> > > > mappings.
> > > >
> > > > 2. If we DON'T have another instantiated service, but DO have a
> service
> > > > template that could fit the bill, perhaps we need to instantiate that
> > > other
> > > > service first. One obvious option is to do this automatically. But I
> > feel
> > > > like this can create unforeseen consequences -- for example, some
> dummy
> > > > test template that someone happened to have in the database might get
> > > > instantiated by mistake. Also, it might need to trigger multiple
> > install
> > > > workflows at once... a big mess. So I suggest that instead we
> provide a
> > > > very detailed validation error here saying that the requirement
> cannot
> > be
> > > > satisfied, HOWEVER there exist service templates A, B, and C that can
> > > > substitute for us, so maybe the nice user would like to instantiate
> > them
> > > > first? This seems very reasonable to me.
> > > >
> > > > 3. If indeed another service satisfies this, a special node is added
> to
> > > the
> > > > current service (with the correct type -- but without a service
> > template
> > > > foreign key), which serves as a proxy of the other service template.
> > I'm
> > > > not sure how we would mark this exactly. We can't use the service_fk
> > > field,
> > > > because it's still in our current service. So perhaps there's need
> of a
> > > new
> > > > fk field, maybe substituted_service_fk?
> > > >
> > > > The above might be "sensible defaults," but it seems to me that users
> > > > really need control over this. So I propose to add a new
> > aria.Composition
> > > > policy that would let you provide hints for this mechanism. For
> > example,
> > > > you might want to "filter" the target service by service template
> name
> > > and
> > > > even by metadata in the service template. For example, you might want
> > to
> > > > require version 1.2.2 of a specific service, no less.
> > > >
> > > > Those are some quick thoughts. Exactly how such a policy would look
> > with
> > > > require more thought...
> > > >
> > > >
> > > > On Tue, Aug 1, 2017 at 2:20 PM, Avia Efrat  wrote:
> > > >
> > > > > Hello all,
> > > > >
> > > > > I'm starting to work on a full implementation of
> > substitution_mapping,
> > > > > which will lead to the ability of service composition.
> > > > >
> > > > > For those unacquainted with substitution mapping, here are some
> quick
> > > > > resources:
> > > > > *From the spec
> > > > >  > > > > YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html>,
> > > > > sections:*
> > > > > 2.10
> > > > >  > 

Re: Service Composition / Substitution Mapping

[DeWayne Filppi]

Cool.  Missed that.  That leaves things almost completely wide open from
the orchestrator side, IOW few predefined keys.  Too few IMHO, but if
everyone uses ARIA conventions it could work.

On Tue, Aug 1, 2017 at 11:49 PM, Tal Liron  wrote:

> I agree! Luckily metadata exists in the 1.0 spec. :)
>
> http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-
> YAML/v1.0/cos01/TOSCA-Simple-Profile-YAML-v1.0-cos01.html#_Toc379455044
>
> On Tue, Aug 1, 2017 at 7:16 PM, DeWayne Filppi 
> wrote:
>
> > It occurs that it might be useful to be able to tag service templates
> with
> > arbitrary meta-data.  Perhaps at one level carried forward from a CSAR
> > manifest, but also user definable.  This would allow inter-service
> > references to be definitive, if desired.  This could be implicitly
> defined
> > as a capability by the orchestrator, but some kind of special requirement
> > type(s) would be needed to utilize it.  This way, external repos could be
> > used safely and directly without the separate load step.
> >
> > On Tue, Aug 1, 2017 at 12:43 PM, Tal Liron  wrote:
> >
> > > Thanks for the kudos. :)
> > >
> > > This topic was discussed on this list a while ago. It's indeed tricky
> to
> > > get right, because TOSCA leaves a lot of room for the orchestrator to
> > > implement.
> > >
> > > I'm thinking of it working something like this:
> > >
> > > 1. The reqs-and-caps engine by default will always look for satisfiable
> > > capabilities within the currently instantiated service. HOWEVER, if
> such
> > a
> > > capability is not present, the option is there to look for another
> > > instantiated service that exposes the capabilities in substitution
> > > mappings.
> > >
> > > 2. If we DON'T have another instantiated service, but DO have a service
> > > template that could fit the bill, perhaps we need to instantiate that
> > other
> > > service first. One obvious option is to do this automatically. But I
> feel
> > > like this can create unforeseen consequences -- for example, some dummy
> > > test template that someone happened to have in the database might get
> > > instantiated by mistake. Also, it might need to trigger multiple
> install
> > > workflows at once... a big mess. So I suggest that instead we provide a
> > > very detailed validation error here saying that the requirement cannot
> be
> > > satisfied, HOWEVER there exist service templates A, B, and C that can
> > > substitute for us, so maybe the nice user would like to instantiate
> them
> > > first? This seems very reasonable to me.
> > >
> > > 3. If indeed another service satisfies this, a special node is added to
> > the
> > > current service (with the correct type -- but without a service
> template
> > > foreign key), which serves as a proxy of the other service template.
> I'm
> > > not sure how we would mark this exactly. We can't use the service_fk
> > field,
> > > because it's still in our current service. So perhaps there's need of a
> > new
> > > fk field, maybe substituted_service_fk?
> > >
> > > The above might be "sensible defaults," but it seems to me that users
> > > really need control over this. So I propose to add a new
> aria.Composition
> > > policy that would let you provide hints for this mechanism. For
> example,
> > > you might want to "filter" the target service by service template name
> > and
> > > even by metadata in the service template. For example, you might want
> to
> > > require version 1.2.2 of a specific service, no less.
> > >
> > > Those are some quick thoughts. Exactly how such a policy would look
> with
> > > require more thought...
> > >
> > >
> > > On Tue, Aug 1, 2017 at 2:20 PM, Avia Efrat  wrote:
> > >
> > > > Hello all,
> > > >
> > > > I'm starting to work on a full implementation of
> substitution_mapping,
> > > > which will lead to the ability of service composition.
> > > >
> > > > For those unacquainted with substitution mapping, here are some quick
> > > > resources:
> > > > *From the spec
> > > >  > > > YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html>,
> > > > sections:*
> > > > 2.10
> > > >  > > > YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725208>,
> > > > 2.11
> > > >  > > > YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725209>
> > > > (theory and examples)
> > > > 3.8.1, 3.8.2 (grammar)
> > > > *From Tal's amazing lecture on TOSCA
> > > > :*
> > > > 00:00 until 12:30.
> > > >
> > > > If anyone wishes to:
> > > > * ask questions regarding this feature
> > > > * suggest real-life use cases
> > > > * offer their insight about vague parts of the spec
> > > > * anything else about substitution mapping and service composition
> > > > Then please, feel encouraged to leave your feedback!
> 

Re: Service Composition / Substitution Mapping

[Tal Liron]

I agree! Luckily metadata exists in the 1.0 spec. :)

http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/cos01/TOSCA-Simple-Profile-YAML-v1.0-cos01.html#_Toc379455044

On Tue, Aug 1, 2017 at 7:16 PM, DeWayne Filppi  wrote:

> It occurs that it might be useful to be able to tag service templates with
> arbitrary meta-data.  Perhaps at one level carried forward from a CSAR
> manifest, but also user definable.  This would allow inter-service
> references to be definitive, if desired.  This could be implicitly defined
> as a capability by the orchestrator, but some kind of special requirement
> type(s) would be needed to utilize it.  This way, external repos could be
> used safely and directly without the separate load step.
>
> On Tue, Aug 1, 2017 at 12:43 PM, Tal Liron  wrote:
>
> > Thanks for the kudos. :)
> >
> > This topic was discussed on this list a while ago. It's indeed tricky to
> > get right, because TOSCA leaves a lot of room for the orchestrator to
> > implement.
> >
> > I'm thinking of it working something like this:
> >
> > 1. The reqs-and-caps engine by default will always look for satisfiable
> > capabilities within the currently instantiated service. HOWEVER, if such
> a
> > capability is not present, the option is there to look for another
> > instantiated service that exposes the capabilities in substitution
> > mappings.
> >
> > 2. If we DON'T have another instantiated service, but DO have a service
> > template that could fit the bill, perhaps we need to instantiate that
> other
> > service first. One obvious option is to do this automatically. But I feel
> > like this can create unforeseen consequences -- for example, some dummy
> > test template that someone happened to have in the database might get
> > instantiated by mistake. Also, it might need to trigger multiple install
> > workflows at once... a big mess. So I suggest that instead we provide a
> > very detailed validation error here saying that the requirement cannot be
> > satisfied, HOWEVER there exist service templates A, B, and C that can
> > substitute for us, so maybe the nice user would like to instantiate them
> > first? This seems very reasonable to me.
> >
> > 3. If indeed another service satisfies this, a special node is added to
> the
> > current service (with the correct type -- but without a service template
> > foreign key), which serves as a proxy of the other service template. I'm
> > not sure how we would mark this exactly. We can't use the service_fk
> field,
> > because it's still in our current service. So perhaps there's need of a
> new
> > fk field, maybe substituted_service_fk?
> >
> > The above might be "sensible defaults," but it seems to me that users
> > really need control over this. So I propose to add a new aria.Composition
> > policy that would let you provide hints for this mechanism. For example,
> > you might want to "filter" the target service by service template name
> and
> > even by metadata in the service template. For example, you might want to
> > require version 1.2.2 of a specific service, no less.
> >
> > Those are some quick thoughts. Exactly how such a policy would look with
> > require more thought...
> >
> >
> > On Tue, Aug 1, 2017 at 2:20 PM, Avia Efrat  wrote:
> >
> > > Hello all,
> > >
> > > I'm starting to work on a full implementation of substitution_mapping,
> > > which will lead to the ability of service composition.
> > >
> > > For those unacquainted with substitution mapping, here are some quick
> > > resources:
> > > *From the spec
> > >  > > YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html>,
> > > sections:*
> > > 2.10
> > >  > > YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725208>,
> > > 2.11
> > >  > > YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725209>
> > > (theory and examples)
> > > 3.8.1, 3.8.2 (grammar)
> > > *From Tal's amazing lecture on TOSCA
> > > :*
> > > 00:00 until 12:30.
> > >
> > > If anyone wishes to:
> > > * ask questions regarding this feature
> > > * suggest real-life use cases
> > > * offer their insight about vague parts of the spec
> > > * anything else about substitution mapping and service composition
> > > Then please, feel encouraged to leave your feedback!
> > >
> >
>

Re: Service Composition / Substitution Mapping

[DeWayne Filppi]

It occurs that it might be useful to be able to tag service templates with
arbitrary meta-data.  Perhaps at one level carried forward from a CSAR
manifest, but also user definable.  This would allow inter-service
references to be definitive, if desired.  This could be implicitly defined
as a capability by the orchestrator, but some kind of special requirement
type(s) would be needed to utilize it.  This way, external repos could be
used safely and directly without the separate load step.

On Tue, Aug 1, 2017 at 12:43 PM, Tal Liron  wrote:

> Thanks for the kudos. :)
>
> This topic was discussed on this list a while ago. It's indeed tricky to
> get right, because TOSCA leaves a lot of room for the orchestrator to
> implement.
>
> I'm thinking of it working something like this:
>
> 1. The reqs-and-caps engine by default will always look for satisfiable
> capabilities within the currently instantiated service. HOWEVER, if such a
> capability is not present, the option is there to look for another
> instantiated service that exposes the capabilities in substitution
> mappings.
>
> 2. If we DON'T have another instantiated service, but DO have a service
> template that could fit the bill, perhaps we need to instantiate that other
> service first. One obvious option is to do this automatically. But I feel
> like this can create unforeseen consequences -- for example, some dummy
> test template that someone happened to have in the database might get
> instantiated by mistake. Also, it might need to trigger multiple install
> workflows at once... a big mess. So I suggest that instead we provide a
> very detailed validation error here saying that the requirement cannot be
> satisfied, HOWEVER there exist service templates A, B, and C that can
> substitute for us, so maybe the nice user would like to instantiate them
> first? This seems very reasonable to me.
>
> 3. If indeed another service satisfies this, a special node is added to the
> current service (with the correct type -- but without a service template
> foreign key), which serves as a proxy of the other service template. I'm
> not sure how we would mark this exactly. We can't use the service_fk field,
> because it's still in our current service. So perhaps there's need of a new
> fk field, maybe substituted_service_fk?
>
> The above might be "sensible defaults," but it seems to me that users
> really need control over this. So I propose to add a new aria.Composition
> policy that would let you provide hints for this mechanism. For example,
> you might want to "filter" the target service by service template name and
> even by metadata in the service template. For example, you might want to
> require version 1.2.2 of a specific service, no less.
>
> Those are some quick thoughts. Exactly how such a policy would look with
> require more thought...
>
>
> On Tue, Aug 1, 2017 at 2:20 PM, Avia Efrat  wrote:
>
> > Hello all,
> >
> > I'm starting to work on a full implementation of substitution_mapping,
> > which will lead to the ability of service composition.
> >
> > For those unacquainted with substitution mapping, here are some quick
> > resources:
> > *From the spec
> >  > YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html>,
> > sections:*
> > 2.10
> >  > YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725208>,
> > 2.11
> >  > YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725209>
> > (theory and examples)
> > 3.8.1, 3.8.2 (grammar)
> > *From Tal's amazing lecture on TOSCA
> > :*
> > 00:00 until 12:30.
> >
> > If anyone wishes to:
> > * ask questions regarding this feature
> > * suggest real-life use cases
> > * offer their insight about vague parts of the spec
> > * anything else about substitution mapping and service composition
> > Then please, feel encouraged to leave your feedback!
> >
>

Re: Service Composition / Substitution Mapping

[Tal Liron]

Thanks for the kudos. :)

This topic was discussed on this list a while ago. It's indeed tricky to
get right, because TOSCA leaves a lot of room for the orchestrator to
implement.

I'm thinking of it working something like this:

1. The reqs-and-caps engine by default will always look for satisfiable
capabilities within the currently instantiated service. HOWEVER, if such a
capability is not present, the option is there to look for another
instantiated service that exposes the capabilities in substitution mappings.

2. If we DON'T have another instantiated service, but DO have a service
template that could fit the bill, perhaps we need to instantiate that other
service first. One obvious option is to do this automatically. But I feel
like this can create unforeseen consequences -- for example, some dummy
test template that someone happened to have in the database might get
instantiated by mistake. Also, it might need to trigger multiple install
workflows at once... a big mess. So I suggest that instead we provide a
very detailed validation error here saying that the requirement cannot be
satisfied, HOWEVER there exist service templates A, B, and C that can
substitute for us, so maybe the nice user would like to instantiate them
first? This seems very reasonable to me.

3. If indeed another service satisfies this, a special node is added to the
current service (with the correct type -- but without a service template
foreign key), which serves as a proxy of the other service template. I'm
not sure how we would mark this exactly. We can't use the service_fk field,
because it's still in our current service. So perhaps there's need of a new
fk field, maybe substituted_service_fk?

The above might be "sensible defaults," but it seems to me that users
really need control over this. So I propose to add a new aria.Composition
policy that would let you provide hints for this mechanism. For example,
you might want to "filter" the target service by service template name and
even by metadata in the service template. For example, you might want to
require version 1.2.2 of a specific service, no less.

Those are some quick thoughts. Exactly how such a policy would look with
require more thought...


On Tue, Aug 1, 2017 at 2:20 PM, Avia Efrat  wrote:

> Hello all,
>
> I'm starting to work on a full implementation of substitution_mapping,
> which will lead to the ability of service composition.
>
> For those unacquainted with substitution mapping, here are some quick
> resources:
> *From the spec
>  YAML/v1.0/TOSCA-Simple-Profile-YAML-v1.0.html>,
> sections:*
> 2.10
>  YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725208>,
> 2.11
>  YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725209>
> (theory and examples)
> 3.8.1, 3.8.2 (grammar)
> *From Tal's amazing lecture on TOSCA
> :*
> 00:00 until 12:30.
>
> If anyone wishes to:
> * ask questions regarding this feature
> * suggest real-life use cases
> * offer their insight about vague parts of the spec
> * anything else about substitution mapping and service composition
> Then please, feel encouraged to leave your feedback!
>

Service Composition / Substitution Mapping

[Avia Efrat]

Hello all,

I'm starting to work on a full implementation of substitution_mapping,
which will lead to the ability of service composition.

For those unacquainted with substitution mapping, here are some quick
resources:
*From the spec
,
sections:*
2.10
,
2.11

(theory and examples)
3.8.1, 3.8.2 (grammar)
*From Tal's amazing lecture on TOSCA
:*
00:00 until 12:30.

If anyone wishes to:
* ask questions regarding this feature
* suggest real-life use cases
* offer their insight about vague parts of the spec
* anything else about substitution mapping and service composition
Then please, feel encouraged to leave your feedback!