In my opinion, the new composite service should indeed be a single service,
so the ARIA CLI will show it as one (if a substituting service already
existed, it would be "dissolved" into the new top-level service). The
composition will show its traces if you look more deeply, because you'll
see that some node templates come from a different service template.
Perhaps our CLI can detect this and tell the user which service templates
were used to create the nodes in current service.
The rest of what you described I think is not related directly to service
composition, but resource realization. The way we currently handle this is
by being very explicit: you have to use the derived Compute types included
in our AWS or Openstack plugins, for example.
I think we should still keep this, because sometimes you want to be very
explicit (and use specific features of the platform), but I actually have
some ideas for multi-VIM support that would work differently: the idea is
to use the basic Compute type, with detailed properties in its Container
and OperatingSystem capabilities. There would then be a policy, perhaps
aria.Realization, that would hint at what kind of Compute nodes you are
looking for (you can apply the policy to specific nodes or groups, or
generally). It might be possible to still have strong platform-specific
feature support here, perhaps by implementing a general string key-value
dict that could include hints for the specific plugin to use.
Anyway, just ideas at this point...
On Fri, Aug 11, 2017 at 2:25 PM, DeWayne Filppi <dewa...@cloudify.co> wrote:
> Good stuff. Obviously a "fancy include" was the wrong metaphor. The
> lifecycles are linked though. When a create the referencing service (aria
> service create), ARIA will match the other service template, and run create
> service on it, and then continue on. Uninstall of the referencer will
> destroy the referenced service. After such an instantiation, would listing
> all services produce two (perhaps with some parent indicator) or one? From
> what I've seen, reqs and caps don't have the equivalent of a lifecycle
> interface associated. Maybe I missed it. It is implicit in the
> orchestrator to match stuff up at service creation time, I'm assuming, and
> using plugins somehow? The sort of flagship scenario is matching cloud
> compute resources without specifying an image. Since discovering whether a
> particular cloud has a given capability, I guessing a plugin for that cloud
> has to search cloud inventory and perhaps quota to provide info for the
> match. Since plugins (at least the Cloudify variety) tend to be triggered
> due to operations that they connect to lifecycle methods, and there is no
> "materialize_capabilities" interface that I noticed, some other kind of
> magic must be performed. Maybe this is an orchestrator-specific detail
> outside of TOSCA.
>
> DeWayne
>
> On Fri, Aug 11, 2017 at 10:57 AM, Tal Liron <t...@cloudify.co> wrote:
>
> > OK, let me try to organize this differently. Three potential conceptions
> > here:
> >
> > 1) A "fancy include," as you say. All that would happen here is that the
> > TOSCA parser would automatically find the service template to include
> from
> > some kind of repository of recognized service templates, and just include
> > that. The language in the TOSCA spec suggests that this is not the
> > intention: it is something that happens at the "orchestrator," not the
> > "parser."
> >
> > 2) Static service composition. This happens not at the parsing stage, but
> > rather the instantiation stage, where reqs-and-caps happens. I think this
> > is what is intended: substitution mapping is specifically about mapping
> > reqs-and-caps. And this is also where things like scalability and
> placement
> > happen: think of a requirement matching a capability, but that capability
> > not having enough capacity. So, the node might be able to scale out: in
> the
> > case of substitution, this would mean duplicating an entire service
> > instance. My understanding is that this is what is intended by TOSCA, and
> > it's entirely within the scope of what we can do. We've recently just
> > refactored the instantiation phase into a new "topology" module where
> > exactly all this logic is concentrated. So think of it this way -- ARIA
> has
> > three parts: "parser," "topology manager," and "workflow engine"
> > ("orchestrator").
> >
> > (I think I might have confused some thing here a bit when mentioning a
> > "logical proxy node." I do not mean an actual piece of proxy software
> > running on a machine somewhere. I mean just a data point in the
> > ARIA-generated topology that can be used as a barrier of sorts when
> > constructing the task graph -- because the task graph follows
> > relationships, the edges of the topology. It could be that we discover in
> > our POC that this is not needed, because actually the substitution node
> is
> > already part of our topology data model and we might be able to easily
> take
> > that into account when generating the task graph.)
> >
> > 3) Live service composition. I think this is a fantasy of some people:
> that
> > ARIA would be able to take existing, "running" service chains and run
> > workflows with them. I do think this is a solvable problem, but not via
> > substitution mapping per se. A solution could involve deployment of a
> proxy
> > service (which could actually be encapsulated in a substitution mapping,
> > but doesn't have to be), or configuring specialized virtual ports via an
> > SDN controller, or via logical proxy nodes created via an inspection
> tool,
> > etc. I cannot believe that there is a one-size-fits-all solution to this
> > problem. The dream of NFV might be to have everything connect to each
> other
> > like LEGO blocks, but there are far too many protocols, configuration,
> and
> > security standards. I think point #2, what I called "static service
> > composition," is a realistic compromise and within the scope of what
> TOSCA
> > promises.
> >
> >
> > On Fri, Aug 11, 2017 at 12:08 PM, DeWayne Filppi <dewa...@cloudify.co>
> > wrote:
> >
> > > To my eyes, the spec doesn't speak of runtime service substituion, but
> > > parsetime template composition. IOW, substitution mapping is a fancy
> > > "include", or the equivalent of an interface definition. Is it
> > understood
> > > by the ARIA team that this includes proxying of running services? IOW,
> > if
> > > my template requires a database service that my template does *not*
> want
> > to
> > > control the lifecycle of, I can "substitution map" an instance of a
> > > template (i.e. a running service)? This would be a lovely feature, but
> > > it's not really a "substitution map", rather more of a "service proxy"
> > (as
> > > implemented as a plugin in Cloudify). Just trying to clarify. Maybe
> > the
> > > community thinks that "substitution map" as something that occurs
> beyond
> > > parsetime, or should.
> > >
> > > On Fri, Aug 11, 2017 at 9:52 AM, Tal Liron <t...@cloudify.co> wrote:
> > >
> > > > Well, DJ, it's all just opinion at this stage and feedback is
> welcome!
> > > >
> > > > Option 1:
> > > > > To look at satisfying nodes present in a substituting
> > service,
> > > > > Have these nodes part of the newly created service and remove the
> > > > > substituting service(nodes with different ID's. Also we are very
> much
> > > in
> > > > > favor of UUID )
> > > > > With this approach I guess the substituting service should
> > not
> > > > > have any associated workflows running. If at all an workflow
> > execution
> > > is
> > > > > already triggered I hope this service will not be considered for
> > > > > substitution.
> > > > > I hope this is the correct approach when we are looking at
> a
> > > > > service for the substitution
> > > > >
> > > >
> > > > Yes, this is a good idea. It would be easy to discover this according
> > to
> > > > the stored node state -- just make sure that all nodes are in a
> stable
> > > > state before composition.
> > > >
> > > > This leads to a general issue: before composition, the substituting
> > > > services must be validated in some way before composition begins.
> > > >
> > > > Also, let's all start using TOSCA terminology here: the containing
> > > service
> > > > is called the "top-level service," and the contained services are
> > called
> > > > "substituting services."
> > > >
> > > > Also, we keep using trivial examples, but it's definitely possible
> for
> > a
> > > > top-level service to require several substituting services at the
> same
> > > > time. I can definitely see such things happening in NFV with even
> > simple
> > > > service chains. Basically every VNF could be a substituting service.
> > > >
> > > > So, actually one of the validations would be to make sure you do not
> > > create
> > > > circular composition: if the top-level also has substitution
> mappings,
> > > you
> > > > need to make sure that one of the substituting ones doesn't require
> it.
> > > :)
> > > > Not a very likely situation, but it would lead to failure.
> > > >
> > > >
> > > >
> > > > > Option 2:
> > > > > While creating a service look at the req-caps at the
> > > > > service-template level and create a service including the nodes
> > > provided
> > > > by
> > > > > the substituting service-template. With this approach there would
> not
> > > be
> > > > > any service created from the service-template which is providing
> > the
> > > > > substitution functionality. The service-template would remain the
> > same
> > > > but
> > > > > only the service would be added with extra nodes.
> > > > >
> > > > > Are you considering both option 1 & 2 for the implementation ? If
> not
> > > > > which one do you feel which take priority. I see option 2 at this
> > stage
> > > > > could be the best possible approach
> > > > > Also could you please let me know a tentative time for this feature
> > to
> > > be
> > > > > available?
> > > > >
> > > >
> > > > I think both options 1 and 2 make sense and are useful, and actually
> > one
> > > is
> > > > a subset of the other.
> > > >
> > > > With option #2 (substituting a service template), it means new nodes
> > are
> > > > instantiated and the composed service would include all nodes. So, an
> > > > "install" workflow would install everything at once. In this case we
> do
> > > > need to fix the lifecycle workflows to be "boundary aware," so that
> > > > workflows of substituting service nodes are part of their own task
> > > graph. I
> > > > think that possibly using a logical proxy node in between might solve
> > > this
> > > > situation automatically.
> > > >
> > > > With option #1 (substituting a service) the substituting nodes might
> > > > already be installed. Or not (they might have been instantiated but
> > still
> > > > not installed). So, the lifecycle workflows should only work on the
> > nodes
> > > > that have not yet been installed.
> > > >
> > > > The point is that we just need to beef up the lifecycle workflows to
> > > > properly work with boundaries and with a mix of nodes in different
> > > states.
> > > > If they can do that, then they can handle any kind of service
> > > composition,
> > > > whether it's option #1 or option #2.
> > > >
> > > > I don't think we can provide a timeline yet, but I will say that we
> are
> > > in
> > > > the research phase and may have a POC soon. Avia is in charge of this
> > > JIRA,
> > > > so I will let him chime in with the current state of things.
> > > >
> > >
> >
>
