Re: [openstack-dev] Incubation Request: Murano

[Thierry Carrez]

Anne Gentle wrote:
>> The reference list lives in the governance git repository:
>> 
>> 
>> http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml
> 
> A bit of metadata I'd like added to the programs.yaml file is which
> release the project was in what status, integrated or incubated. Shall I
> propose a patch?
> 
> Currently you have to look at italicizations
> on https://wiki.openstack.org/wiki/Programs to determine
> integrated/incubated but you can't really know what release.

Yes, this has been requested in the past. Feel free to propose a patch.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Steven Dake]

On 03/07/2014 07:28 AM, Anne Gentle wrote:



On Fri, Mar 7, 2014 at 2:53 AM, Thierry Carrez > wrote:


Steven Dake wrote:
> I'm a bit confused as well as to how a incubated project would be
> differentiated from a integrated project in one program.  This
may have
> already been discussed by the TC.  For example, Red Hat doesn't
> officially support incubated projects, but we officially support
(with
> our full sales/training/documentation/support/ plus a whole bunch of
> other Red Hat internalisms) Integrated projects.  OpenStack
vendors need
> a way to let customers know (through an upstream page?) what a
project
> in a specific program's status is so we can appropriately set
> expectations with the community and  customers.

The reference list lives in the governance git repository:


http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml


A bit of metadata I'd like added to the programs.yaml file is which 
release the project was in what status, integrated or incubated. Shall 
I propose a patch?


Currently you have to look at italicizations on 
https://wiki.openstack.org/wiki/Programs to determine 
integrated/incubated but you can't really know what release.


Anne

Anne,

I think from a vendor POV, this makes great sense for OpenStack vendors 
to prevent a "the programs list says its integrated" when it wasn't in 
the particular version of OpenStack being used.


Regards
-steve


--
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Steven Dake]

On 03/07/2014 01:53 AM, Thierry Carrez wrote:

Steven Dake wrote:

I'm a bit confused as well as to how a incubated project would be
differentiated from a integrated project in one program.  This may have
already been discussed by the TC.  For example, Red Hat doesn't
officially support incubated projects, but we officially support (with
our full sales/training/documentation/support/ plus a whole bunch of
other Red Hat internalisms) Integrated projects.  OpenStack vendors need
a way to let customers know (through an upstream page?) what a project
in a specific program's status is so we can appropriately set
expectations with the community and  customers.

The reference list lives in the governance git repository:

http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml


Thierry,

Thanks that is a great reference.

Regards
-steve


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Anne Gentle]

On Fri, Mar 7, 2014 at 2:53 AM, Thierry Carrez wrote:

> Steven Dake wrote:
> > I'm a bit confused as well as to how a incubated project would be
> > differentiated from a integrated project in one program.  This may have
> > already been discussed by the TC.  For example, Red Hat doesn't
> > officially support incubated projects, but we officially support (with
> > our full sales/training/documentation/support/ plus a whole bunch of
> > other Red Hat internalisms) Integrated projects.  OpenStack vendors need
> > a way to let customers know (through an upstream page?) what a project
> > in a specific program's status is so we can appropriately set
> > expectations with the community and  customers.
>
> The reference list lives in the governance git repository:
>
>
> http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml
>
>
A bit of metadata I'd like added to the programs.yaml file is which release
the project was in what status, integrated or incubated. Shall I propose a
patch?

Currently you have to look at italicizations on
https://wiki.openstack.org/wiki/Programs to determine integrated/incubated
but you can't really know what release.

Anne


> --
> Thierry Carrez (ttx)
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Thierry Carrez]

Steven Dake wrote:
> I'm a bit confused as well as to how a incubated project would be
> differentiated from a integrated project in one program.  This may have
> already been discussed by the TC.  For example, Red Hat doesn't
> officially support incubated projects, but we officially support (with
> our full sales/training/documentation/support/ plus a whole bunch of
> other Red Hat internalisms) Integrated projects.  OpenStack vendors need
> a way to let customers know (through an upstream page?) what a project
> in a specific program's status is so we can appropriately set
> expectations with the community and  customers.

The reference list lives in the governance git repository:

http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Steven Dake]

On 03/06/2014 03:15 AM, Thierry Carrez wrote:

Steven Dake wrote:

My general take is workflow would fit in the Orchestration program, but
not be integrated into the heat repo specifically.  It would be a
different repo, managed by the same orchestration program just as we
have heat-cfntools and other repositories.  Figuring out how to handle
the who is the core team of people responsible for program's individual
repositories is the most difficult aspect of making such a merge.  For
example, I'd not desire a bunch of folks from Murano +2/+A heat specific
repos until they understood the code base in detail, or atleast the
broad architecture.   I think the same think applies in reverse from the
Murano perspective.  Ideally folks that are core on a specific program
would need to figure out how to learn how to broadly review each repo
(meaning the heat devs would have to come up to speed on murano and
murano devs would have to come up to speed on heat.  Learning a new code
base is a big commitment for an already overtaxed core team.

Being in the same program means you share the same team and PTL, not
necessarily that all projects under the program have the same core
review team. So you could have different core reviewers for both
(although I'd encourage the core for ones become core for the other,
since it will facilitate behaving like a coherent team). You could also
have a single core team with clear expectations set ("do not approve
changes for code you're not familiar with").

This may be possible with jenkins permissions, but what I'd like to see 
is for a way for people familiar with each specific project to be 
graduated to core for that project.  (eg heat or workflow).  An implicit 
expectation do not approve doesn't  totally fit, because at some point, 
we may want to give those folks the ability to approve via a core 
nomination (because they have met the core requirements) for either heat 
or workflow.  WIthout a way of nominating for core for a specific 
project (within a specific program), the poor developer has no way to 
know when they have officially been recognized by the core team as an 
actual core member.


I agree folks in one program need to behave as a coherent team for the 
Orchestration program to be successful, which means a big commitment 
from the existing orchestration program core members (currently 
heat-core) to come up to speed on the example workflow code base and 
community (and vice-versa).


I'm a bit confused as well as to how a incubated project would be 
differentiated from a integrated project in one program.  This may have 
already been discussed by the TC.  For example, Red Hat doesn't 
officially support incubated projects, but we officially support (with 
our full sales/training/documentation/support/ plus a whole bunch of 
other Red Hat internalisms) Integrated projects.  OpenStack vendors need 
a way to let customers know (through an upstream page?) what a project 
in a specific program's status is so we can appropriately set 
expectations with the community and  customers.


Regards
-steve


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Thierry Carrez]

Steven Dake wrote:
> My general take is workflow would fit in the Orchestration program, but
> not be integrated into the heat repo specifically.  It would be a
> different repo, managed by the same orchestration program just as we
> have heat-cfntools and other repositories.  Figuring out how to handle
> the who is the core team of people responsible for program's individual
> repositories is the most difficult aspect of making such a merge.  For
> example, I'd not desire a bunch of folks from Murano +2/+A heat specific
> repos until they understood the code base in detail, or atleast the
> broad architecture.   I think the same think applies in reverse from the
> Murano perspective.  Ideally folks that are core on a specific program
> would need to figure out how to learn how to broadly review each repo
> (meaning the heat devs would have to come up to speed on murano and
> murano devs would have to come up to speed on heat.  Learning a new code
> base is a big commitment for an already overtaxed core team.

Being in the same program means you share the same team and PTL, not
necessarily that all projects under the program have the same core
review team. So you could have different core reviewers for both
(although I'd encourage the core for ones become core for the other,
since it will facilitate behaving like a coherent team). You could also
have a single core team with clear expectations set ("do not approve
changes for code you're not familiar with").

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Georgy Okrokvertskhov]

Hi Steve,

Thank you for sharing your thoughts. I believe that is what we were trying
to receive as a feedback form TC. The current definition of program
actually suggest the scenario you described. A new project will appear
under Orchestration umbrella. Let say there will be two project one is Heat
and another is Workflow (no specific name here, probably some part of
Murano).  Program will have one PTL (current Heat PTL) and two separate
code team for each project. That was our understanding of what we want. I
am not sure that this was enough stressed out on TC meeting. There were no
any intentions to add anything to Heat. Not at all. We just discussed a
possibility to split current Murano App Catalog into two parts. Catalog
part would go to Catalog program to land App Catalog code near Glance
project and integrate them as Glance will store application packages for
Murano App Catalog service. The second part of Murano related to
environment processing (deployment, life cycle management, events) would go
to Orchestration program as a new project like Murano workflows or Murano
environment control or anything else.

As I mentioned in one of the previous e-mails, we already discussed with
the heat team workflows & Heat before HK summit. We understand this very
well that workflows will not fit Heat and we perfectly understand reasons
why.

I think that the good result of last TC was the official mandate to discuss
alignment and integration between projects Glance, Heat, Murano and
probably other projects. Right now we consider the following:
1) Continue discussion around Catalog program mission and how Murano App
Catalog will fit into this program.
2) Start conversation with the Heat team in two directions:
  a) TOSCA and its implementation. How Murano can extend TOSCA and how
TOSCA can help Murano to define an application package. Murano should reuse
as much as possible from TOSCA to implement this open standard
  b) Define the alignment between Heat and Murano. How workflows can
coexist with HOT. What will be the best way to develop both Heat and
Workflows within Orchestration program.
3) Explore Application space for OpenStack. As Thierry mentioned on TC
meeting, there are concerns that it is probably to early for OpenStack to
make a new step up to the stack.

Thanks,
Georgy


On Wed, Mar 5, 2014 at 7:47 PM, Steven Dake  wrote:

> On 03/05/2014 02:16 AM, Thomas Spatzier wrote:
>
>> Georgy Okrokvertskhov  wrote on 05/03/2014
>> 00:32:08:
>>
>>  From: Georgy Okrokvertskhov 
>>> To: "OpenStack Development Mailing List (not for usage questions)"
>>> 
>>> Date: 05/03/2014 00:34
>>> Subject: Re: [openstack-dev] Incubation Request: Murano
>>>
>>> Hi Thomas, Zane,
>>>
>>> Thank you for bringing TOSCA to the discussion. I think this is
>>> important topic as it will help to find better alignment or even
>>> future merge of Murano DSL and Heat templates. Murano DSL uses YAML
>>> representation too, so we can easily merge use constructions from
>>> Heat and probably any other YAML based TOSCA formats.
>>>
>>> I will be glad to join TOSCA TC. Is there any formal process for that?
>>>
>> The first part is that your company must be a member of OASIS. If that is
>> the case, I think you can simply go to the TC page [1] and click a button
>> to join the TC. If your company is not yet a member, you could get in
>> touch
>> with the TC chairs Paul Lipton and Simon Moser and ask for the best next
>> steps. We recently had people from GigaSpaces join the TC, and since they
>> are also doing very TOSCA aligned implementation in Cloudify, their input
>> will probably help a lot to advance TOSCA.
>>
>>  I also would like to use this opportunity and start conversation
>>> with Heat team about Heat roadmap and feature set. As Thomas
>>> mentioned in his previous e-mail TOSCA topology story is quite
>>> covered by HOT. At the same time there are entities like Plans which
>>> are covered by Murano. We had discussion about bringing workflows to
>>> Heat engine before HK summit and it looks like that Heat team has no
>>> plans to bring workflows into Heat. That is actually why we
>>> mentioned Orchestration program as a potential place for Murano DSL
>>> as Heat+Murano together will cover everything which is defined by TOSCA.
>>>
>> I remember the discussions about whether to bring workflows into Heat or
>> not. My personal opinion is that workflows are probably out of the scope
>> of
>> Heat (i.e. everything but the derived orchestration flows the Heat engine
>> implements). So there could well be a layer on-top of Heat that lets Heat
>> deal with all topology-related declarative bu

Re: [openstack-dev] Incubation Request: Murano

[Steven Dake]

On 03/05/2014 02:16 AM, Thomas Spatzier wrote:

Georgy Okrokvertskhov  wrote on 05/03/2014
00:32:08:


From: Georgy Okrokvertskhov 
To: "OpenStack Development Mailing List (not for usage questions)"

Date: 05/03/2014 00:34
Subject: Re: [openstack-dev] Incubation Request: Murano

Hi Thomas, Zane,

Thank you for bringing TOSCA to the discussion. I think this is
important topic as it will help to find better alignment or even
future merge of Murano DSL and Heat templates. Murano DSL uses YAML
representation too, so we can easily merge use constructions from
Heat and probably any other YAML based TOSCA formats.

I will be glad to join TOSCA TC. Is there any formal process for that?

The first part is that your company must be a member of OASIS. If that is
the case, I think you can simply go to the TC page [1] and click a button
to join the TC. If your company is not yet a member, you could get in touch
with the TC chairs Paul Lipton and Simon Moser and ask for the best next
steps. We recently had people from GigaSpaces join the TC, and since they
are also doing very TOSCA aligned implementation in Cloudify, their input
will probably help a lot to advance TOSCA.


I also would like to use this opportunity and start conversation
with Heat team about Heat roadmap and feature set. As Thomas
mentioned in his previous e-mail TOSCA topology story is quite
covered by HOT. At the same time there are entities like Plans which
are covered by Murano. We had discussion about bringing workflows to
Heat engine before HK summit and it looks like that Heat team has no
plans to bring workflows into Heat. That is actually why we
mentioned Orchestration program as a potential place for Murano DSL
as Heat+Murano together will cover everything which is defined by TOSCA.

I remember the discussions about whether to bring workflows into Heat or
not. My personal opinion is that workflows are probably out of the scope of
Heat (i.e. everything but the derived orchestration flows the Heat engine
implements). So there could well be a layer on-top of Heat that lets Heat
deal with all topology-related declarative business and adds workflow-based
orchestration around it. TOSCA could be a way to describe the respective
overarching models and then hand the different processing tasks to the
right engine to deal with it.
My general take is workflow would fit in the Orchestration program, but 
not be integrated into the heat repo specifically.  It would be a 
different repo, managed by the same orchestration program just as we 
have heat-cfntools and other repositories.  Figuring out how to handle 
the who is the core team of people responsible for program's individual 
repositories is the most difficult aspect of making such a merge.  For 
example, I'd not desire a bunch of folks from Murano +2/+A heat specific 
repos until they understood the code base in detail, or atleast the 
broad architecture.   I think the same think applies in reverse from the 
Murano perspective.  Ideally folks that are core on a specific program 
would need to figure out how to learn how to broadly review each repo 
(meaning the heat devs would have to come up to speed on murano and 
murano devs would have to come up to speed on heat.  Learning a new code 
base is a big commitment for an already overtaxed core team.


I believe expanding our scope in this way would require TC approval.

The main reason I don't want workflow in the heat repo specifically is 
because it adds complication to Heat itself.  We want Heat to be one 
nice tidy small set of code that does one thing really well. This makes 
it easy to improve, easy to deploy, and easy to learn!


These reasons are why, for example, we are continuing to push the 
autoscaling implementation out of Heat and into a separate repository 
over the next 1 to 2 cycles   This on the other hand won't be an 
expansion of scope of the Orchestration program, because we already do 
autoscaling, we just want to make it more consumable.


Regards,
-steve



I think TOSCA initiative can be a great place to collaborate. I
think it will be possible then to use Simplified TOSCA format for
Application descriptions as TOSCA is intended to provide such

descriptions.

Is there a team who are driving TOSCA implementation in OpenStack
community? I feel that such team is necessary.

We started to implement a TOSCA YAML to HOT converter and our team member
Sahdev (IRC spzala) has recently submitted code for a new stackforge
project [2]. This is very initial, but could be a point to collaborate.

[1] https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca
[2] https://github.com/stackforge/heat-translator

Regards,
Thomas


Thanks
Georgy

On Tue, Mar 4, 2014 at 2:36 PM, Thomas Spatzier


wrote:

Excerpt from Zane Bitter's message on 04/03/2014 23:16:21:

From: Zane Bitter 
To: openstack-dev@lists.openstack.org
Date: 04/03/2014 23:20
Subject: Re: [openstack-dev] Incubation Request: Murano

On 04

Re: [openstack-dev] Incubation Request: Murano

[Thomas Spatzier]

Forgot to provide the email addresses of Paul and Simon in my last mail:

paul.lip...@ca.com
smo...@de.ibm.com

Regards,
Thomas

> From: Thomas Spatzier/Germany/IBM@IBMDE
> To: "OpenStack Development Mailing List \(not for usage questions\)"
> 
> Date: 05/03/2014 10:21
> Subject: Re: [openstack-dev] Incubation Request: Murano
>
> Georgy Okrokvertskhov  wrote on 05/03/2014
> 00:32:08:
>
> > From: Georgy Okrokvertskhov 
> > To: "OpenStack Development Mailing List (not for usage questions)"
> > 
> > Date: 05/03/2014 00:34
> > Subject: Re: [openstack-dev] Incubation Request: Murano
> >
> > Hi Thomas, Zane,
> >
> > Thank you for bringing TOSCA to the discussion. I think this is
> > important topic as it will help to find better alignment or even
> > future merge of Murano DSL and Heat templates. Murano DSL uses YAML
> > representation too, so we can easily merge use constructions from
> > Heat and probably any other YAML based TOSCA formats.
> >
> > I will be glad to join TOSCA TC. Is there any formal process for that?
>
> The first part is that your company must be a member of OASIS. If that is
> the case, I think you can simply go to the TC page [1] and click a button
> to join the TC. If your company is not yet a member, you could get in
touch
> with the TC chairs Paul Lipton and Simon Moser and ask for the best next
> steps. We recently had people from GigaSpaces join the TC, and since they
> are also doing very TOSCA aligned implementation in Cloudify, their input
> will probably help a lot to advance TOSCA.
>
> >
> > I also would like to use this opportunity and start conversation
> > with Heat team about Heat roadmap and feature set. As Thomas
> > mentioned in his previous e-mail TOSCA topology story is quite
> > covered by HOT. At the same time there are entities like Plans which
> > are covered by Murano. We had discussion about bringing workflows to
> > Heat engine before HK summit and it looks like that Heat team has no
> > plans to bring workflows into Heat. That is actually why we
> > mentioned Orchestration program as a potential place for Murano DSL
> > as Heat+Murano together will cover everything which is defined by
TOSCA.
>
> I remember the discussions about whether to bring workflows into Heat or
> not. My personal opinion is that workflows are probably out of the scope
of
> Heat (i.e. everything but the derived orchestration flows the Heat engine
> implements). So there could well be a layer on-top of Heat that lets Heat
> deal with all topology-related declarative business and adds
workflow-based
> orchestration around it. TOSCA could be a way to describe the respective
> overarching models and then hand the different processing tasks to the
> right engine to deal with it.
>
> >
> > I think TOSCA initiative can be a great place to collaborate. I
> > think it will be possible then to use Simplified TOSCA format for
> > Application descriptions as TOSCA is intended to provide such
> descriptions.
> >
> > Is there a team who are driving TOSCA implementation in OpenStack
> > community? I feel that such team is necessary.
>
> We started to implement a TOSCA YAML to HOT converter and our team member
> Sahdev (IRC spzala) has recently submitted code for a new stackforge
> project [2]. This is very initial, but could be a point to collaborate.
>
> [1] https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca
> [2] https://github.com/stackforge/heat-translator
>
> Regards,
> Thomas
>
> >
> > Thanks
> > Georgy
> >
>
> > On Tue, Mar 4, 2014 at 2:36 PM, Thomas Spatzier
>  > > wrote:
> > Excerpt from Zane Bitter's message on 04/03/2014 23:16:21:
> > > From: Zane Bitter 
> > > To: openstack-dev@lists.openstack.org
> > > Date: 04/03/2014 23:20
> > > Subject: Re: [openstack-dev] Incubation Request: Murano
> > >
> > > On 04/03/14 00:04, Georgy Okrokvertskhov wrote:
> > > >
> > > It so happens that the OASIS's TOSCA technical committee are working
as
> > > we speak on a "TOSCA Simple Profile" that will hopefully make things
> > > easier to use and includes a YAML representation (the latter is great
> > > IMHO, but the key to being able to do it is the former). Work is
still
> > > at a relatively early stage and in my experience they are very much
> open
> > > to input from implementers.
>
> > Nice, I was probably also writing a mail with this information at about
> the
> > same time :-)
> > And yes, we are very much interested in feedback from implementers a

Re: [openstack-dev] Incubation Request: Murano

[Thomas Spatzier]

Hi Stan,

thanks for sharing your thoughts about Murano and relation to TOSCA. I have
added a few comments below.

> From: Stan Lagun 
> To: "OpenStack Development Mailing List (not for usage questions)"
> 
> Date: 05/03/2014 00:51
> Subject: Re: [openstack-dev] Incubation Request: Murano
>
> Hi all,

> Completely agree with Zane. Collaboration with TOSCA TC is a way to
> go as Murano is very close to TOSCA. Like Murano = 0.9 * TOSCA + UI
> + OpenStack services integration.

> Let me share my thoughts on TOSCA as I read all TOSCA docs and I'm
> also the author of initial Murano DSL design proposal so I can
> probably compare them.

> We initially considered to just implement TOSCA before going with
> own DSL. There was no YAML TOSCA out there at that time, just XML
version.

> So here's why we've wrote our own DSL:

> 1. TOSCA is very complex and verbose. Considering there is no
> production-ready tooling for TOSCA users would have to type all
> those tons of XML tags and namespaces and TOSCA XMLs are really hard
> to read and write. No one gonna do this, especially outside of Java-
> enterprise world

Right, that's why we are doing the simple profile and YAML work now to
overcome those adoption issues.

> 2. TOSCA has no workflow language. TOSCA draft states that the
> language is indeed needed and recommends using BPEL or BPMN for that
matter.

Right, the goal of TOSCA was not to define a new workflow language but to
refer to existing ones. This does not mean, of course that other languages
than BPEL or BPMN cannot be used. We still consider standardization of such
a language out of scope of the TC, but if there is some widely adopted flow
language being implemented, e.g. in the course of Murano, I could imagine
that a community could use such a simpler language in an OpenStack
environment. Ideally though, such a simpler workflow language would be
translatable to a standard language like BPMN (or a subset of it) so those
who have a "real" process engine can consume the flow descriptions.

> Earlier versions of Murano showed that some sort of workflow
> language (declarative, imperative whatever) if absolutely required
> for non-trivial cases. If you don't have workflow language then you
> have to hard-code a lot of knowledge into engine in Python. But the
> whole idea of AppCatalog was that users upload (share) their
> application templates that contain application-specific maintenance/
> deployment code that is run in on common shared server (not in any
> particular VM) and thus capable of orchestrating all activities that
> are taking place on different VMs belonging to given application
> (for complex applications with typical enterprise SOA architecture).
> Besides VMs applications can talk to OpenStack services like Heat,
> Neutron, Trove and 3rd party services (DNS registration, NNTP,
> license activation service etc). Especially with the Heat so that
> application can have its VMs and other IaaS resources. There is a
> similar problem in Heat - you can express most of the basic things
> in HOT but once you need something really complex like accessing
> external API, custom load balancing or anything tricky you need to
> resort to Python and write custom resource plugin. And then you
> required to have root access to engine to install that plugin. This
> is not a solution for Murano as in Murano any user can upload
> application manifest at any time without affecting running system
> and without admin permissions.

> Now going back to TOSCA the problem with TOSCA workflows is they are
> not part of standard. There is no standardized way how BPEL would
> access TOSCA attributes and how 2 systems need to interact. This
> alone makes any 2 TOSCA implementations incompatible with each other
> rendering the whole idea of standard useless. It is not standard if
> there is no compatibility.

We have been working on what we call a "plan portability API" that
describes what APIs a TOSCA container has to support so that portable flows
can access topology information. During the v1.0 time frame, though, we
focused on the declarative part (i.e. the topology model). But, yes I agree
that this part needs to be done so that also plans get portable. If you are
having experience in this area, it would be great to collaborate and see if
we can feed your input into the TOSCA effort.

> And again BPEL is heavy XML language that you don't want to have in
> OpenStack. Trust me, I spent significant time studying it. And if
> there is YAML version of TOSCA that is much more readable than XML
> one there is no such thing for BPEL. And I'm not aware of any
> adequate replacement for it

I agree that BPEL and BPMN are very heavy and hard to use without tooling,
so no objection on looking at a l

Re: [openstack-dev] Incubation Request: Murano

[Thomas Spatzier]

Georgy Okrokvertskhov  wrote on 05/03/2014
00:32:08:

> From: Georgy Okrokvertskhov 
> To: "OpenStack Development Mailing List (not for usage questions)"
> 
> Date: 05/03/2014 00:34
> Subject: Re: [openstack-dev] Incubation Request: Murano
>
> Hi Thomas, Zane,
>
> Thank you for bringing TOSCA to the discussion. I think this is
> important topic as it will help to find better alignment or even
> future merge of Murano DSL and Heat templates. Murano DSL uses YAML
> representation too, so we can easily merge use constructions from
> Heat and probably any other YAML based TOSCA formats.
>
> I will be glad to join TOSCA TC. Is there any formal process for that?

The first part is that your company must be a member of OASIS. If that is
the case, I think you can simply go to the TC page [1] and click a button
to join the TC. If your company is not yet a member, you could get in touch
with the TC chairs Paul Lipton and Simon Moser and ask for the best next
steps. We recently had people from GigaSpaces join the TC, and since they
are also doing very TOSCA aligned implementation in Cloudify, their input
will probably help a lot to advance TOSCA.

>
> I also would like to use this opportunity and start conversation
> with Heat team about Heat roadmap and feature set. As Thomas
> mentioned in his previous e-mail TOSCA topology story is quite
> covered by HOT. At the same time there are entities like Plans which
> are covered by Murano. We had discussion about bringing workflows to
> Heat engine before HK summit and it looks like that Heat team has no
> plans to bring workflows into Heat. That is actually why we
> mentioned Orchestration program as a potential place for Murano DSL
> as Heat+Murano together will cover everything which is defined by TOSCA.

I remember the discussions about whether to bring workflows into Heat or
not. My personal opinion is that workflows are probably out of the scope of
Heat (i.e. everything but the derived orchestration flows the Heat engine
implements). So there could well be a layer on-top of Heat that lets Heat
deal with all topology-related declarative business and adds workflow-based
orchestration around it. TOSCA could be a way to describe the respective
overarching models and then hand the different processing tasks to the
right engine to deal with it.

>
> I think TOSCA initiative can be a great place to collaborate. I
> think it will be possible then to use Simplified TOSCA format for
> Application descriptions as TOSCA is intended to provide such
descriptions.
>
> Is there a team who are driving TOSCA implementation in OpenStack
> community? I feel that such team is necessary.

We started to implement a TOSCA YAML to HOT converter and our team member
Sahdev (IRC spzala) has recently submitted code for a new stackforge
project [2]. This is very initial, but could be a point to collaborate.

[1] https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca
[2] https://github.com/stackforge/heat-translator

Regards,
Thomas

>
> Thanks
> Georgy
>

> On Tue, Mar 4, 2014 at 2:36 PM, Thomas Spatzier
 > wrote:
> Excerpt from Zane Bitter's message on 04/03/2014 23:16:21:
> > From: Zane Bitter 
> > To: openstack-dev@lists.openstack.org
> > Date: 04/03/2014 23:20
> > Subject: Re: [openstack-dev] Incubation Request: Murano
> >
> > On 04/03/14 00:04, Georgy Okrokvertskhov wrote:
> > >
> > It so happens that the OASIS's TOSCA technical committee are working as
> > we speak on a "TOSCA Simple Profile" that will hopefully make things
> > easier to use and includes a YAML representation (the latter is great
> > IMHO, but the key to being able to do it is the former). Work is still
> > at a relatively early stage and in my experience they are very much
open
> > to input from implementers.

> Nice, I was probably also writing a mail with this information at about
the
> same time :-)
> And yes, we are very much interested in feedback from implementers and
open
> to suggestions. If we can find gaps and fill them with good proposals,
now
> is the right time.
>
> >
> > I would strongly encourage you to get involved in this effort (by
> > joining the TOSCA TC), and also to architect Murano in such a way that
> > it can accept input in multiple formats (this is something we are
making
> > good progress toward in Heat). Ideally the DSL format for Murano+Heat
> > should be a trivial translation away from the relevant parts of the
YAML
> > representation of TOSCA Simple Profile.

> Right, having a straight-forward translation would be really desirable.
The
> way to get there can actually be two-fold: (1) any feedback we get from
the
> Murano folks on the TOSCA simple profile and YAML can help us to make

Re: [openstack-dev] Incubation Request: Murano

[Stan Lagun]

Hi all,

Completely agree with Zane. Collaboration with TOSCA TC is a way to go as
Murano is very close to TOSCA. Like Murano = 0.9 * TOSCA + UI + OpenStack
services integration.

Let me share my thoughts on TOSCA as I read all TOSCA docs and I'm also the
author of initial Murano DSL design proposal so I can probably compare them.

We initially considered to just implement TOSCA before going with own DSL.
There was no YAML TOSCA out there at that time, just XML version.

So here's why we've wrote our own DSL:

1. TOSCA is very complex and verbose. Considering there is no
production-ready tooling for TOSCA users would have to type all those tons
of XML tags and namespaces and TOSCA XMLs are really hard to read and
write. No one gonna do this, especially outside of Java-enterprise world

2. TOSCA has no workflow language. TOSCA draft states that the language is
indeed needed and recommends using BPEL or BPMN for that matter.
Earlier versions of Murano showed that some sort of workflow language
(declarative, imperative whatever) if absolutely required for non-trivial
cases. If you don't have workflow language then you have to hard-code a lot
of knowledge into engine in Python. But the whole idea of AppCatalog was
that users upload (share) their application templates that contain
application-specific maintenance/deployment code that is run in on common
shared server (not in any particular VM) and thus capable of orchestrating
all activities that are taking place on different VMs belonging to given
application (for complex applications with typical enterprise SOA
architecture). Besides VMs applications can talk to OpenStack services like
Heat, Neutron, Trove and 3rd party services (DNS registration, NNTP,
license activation service etc). Especially with the Heat so that
application can have its VMs and other IaaS resources. There is a similar
problem in Heat - you can express most of the basic things in HOT but once
you need something really complex like accessing external API, custom load
balancing or anything tricky you need to resort to Python and write custom
resource plugin. And then you required to have root access to engine to
install that plugin. This is not a solution for Murano as in Murano any
user can upload application manifest at any time without affecting running
system and without admin permissions.

Now going back to TOSCA the problem with TOSCA workflows is they are not
part of standard. There is no standardized way how BPEL would access TOSCA
attributes and how 2 systems need to interact. This alone makes any 2 TOSCA
implementations incompatible with each other rendering the whole idea of
standard useless. It is not standard if there is no compatibility.

And again BPEL is heavy XML language that you don't want to have in
OpenStack. Trust me, I spent significant time studying it. And if there is
YAML version of TOSCA that is much more readable than XML one there is no
such thing for BPEL. And I'm not aware of any adequate replacement for it

3. It seems like nobody really using TOSCA. TOSCA standard defines exact
TOSCA package format. TOSCA was designed so that people can share those
packages (CSARs as TOSCA calls them) between various TOSCA implementations.
I've tried to google those packages. It took me like a hour to find even
most trivial CSAR example. And it was on OASIS site.

4. There is no reference TOSCA implementation. No test suite. There is no
way to check your implementation is really TOSCA-compatible. And no one to
ask questions

5. TOSCA is very immature. They didn't even made XML version used and
already working on YAML version that is not compatible with current draft

6. TOSCA is too restrictive and verbose in some areas while leaving a lot
of white spaces in others


So we decided to go with our own DSL that would eliminate all the problems
above. And I personally feel that Murano is how TOSCA should be looked like
if it was designed for OpenStack. Murano is perfectly aligned with Heat and
other OpenStack services and practices. It is very Python-like. It is easy
to read and write once you learn the basics. It is more universal and less
restrictive than TOSCA. You can do much more than you could ever do in
TOSCA. And it is very extensible.

I hope that Murano and its ideas will find a way into OpenStack community :)



On Wed, Mar 5, 2014 at 2:16 AM, Zane Bitter  wrote:

> On 04/03/14 00:04, Georgy Okrokvertskhov wrote:
>
>>
>> First of all let me highlight that Murano DSL was much inspired by
>> TOSCA. We carefully read this standard before our movement to Murano
>> DSL. TOSCA standard has a lot f of very well designed concepts and ideas
>> which we reused in Murano. There is one obvious draw back of TOSCA -
>> very heavy and verbose XML based syntax. Taking into account that
>> OpenStack itself is clearly moving from XML based representations, it
>> will be strange to bring this huge XML monster back on a higher level.
>> Frankly, the current Murano workflows language is 

Re: [openstack-dev] Incubation Request: Murano

[Georgy Okrokvertskhov]

Hi Thomas, Zane,


Thank you for bringing TOSCA to the discussion. I think this is important
topic as it will help to find better alignment or even future merge of
Murano DSL and Heat templates. Murano DSL uses YAML representation too, so
we can easily merge use constructions from Heat and probably any other YAML
based TOSCA formats.

I will be glad to join TOSCA TC. Is there any formal process for that?

I also would like to use this opportunity and start conversation with Heat
team about Heat roadmap and feature set. As Thomas mentioned in his
previous e-mail TOSCA topology story is quite covered by HOT. At the same
time there are entities like Plans which are covered by Murano. We had
discussion about bringing workflows to Heat engine before HK summit and it
looks like that Heat team has no plans to bring workflows into Heat. That
is actually why we mentioned Orchestration program as a potential place for
Murano DSL as Heat+Murano together will cover everything which is defined
by TOSCA.

I think TOSCA initiative can be a great place to collaborate. I think it
will be possible then to use Simplified TOSCA format for Application
descriptions as TOSCA is intended to provide such descriptions.

Is there a team who are driving TOSCA implementation in OpenStack
community? I feel that such team is necessary.

Thanks
Georgy


On Tue, Mar 4, 2014 at 2:36 PM, Thomas Spatzier
wrote:

> Excerpt from Zane Bitter's message on 04/03/2014 23:16:21:
> > From: Zane Bitter 
> > To: openstack-dev@lists.openstack.org
> > Date: 04/03/2014 23:20
> > Subject: Re: [openstack-dev] Incubation Request: Murano
> >
> > On 04/03/14 00:04, Georgy Okrokvertskhov wrote:
> > >
> > It so happens that the OASIS's TOSCA technical committee are working as
> > we speak on a "TOSCA Simple Profile" that will hopefully make things
> > easier to use and includes a YAML representation (the latter is great
> > IMHO, but the key to being able to do it is the former). Work is still
> > at a relatively early stage and in my experience they are very much open
> > to input from implementers.
>
> Nice, I was probably also writing a mail with this information at about the
> same time :-)
> And yes, we are very much interested in feedback from implementers and open
> to suggestions. If we can find gaps and fill them with good proposals, now
> is the right time.
>
> >
> > I would strongly encourage you to get involved in this effort (by
> > joining the TOSCA TC), and also to architect Murano in such a way that
> > it can accept input in multiple formats (this is something we are making
> > good progress toward in Heat). Ideally the DSL format for Murano+Heat
> > should be a trivial translation away from the relevant parts of the YAML
> > representation of TOSCA Simple Profile.
>
> Right, having a straight-forward translation would be really desirable. The
> way to get there can actually be two-fold: (1) any feedback we get from the
> Murano folks on the TOSCA simple profile and YAML can help us to make TOSCA
> capable of addressing the right use cases, and (2) on the other hand make
> sure the implementation goes in a direction that is in line with what TOSCA
> YAML will look like.
>
> >
> > cheers,
> > Zane.
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Georgy Okrokvertskhov
Architect,
OpenStack Platform Products,
Mirantis
http://www.mirantis.com
Tel. +1 650 963 9828
Mob. +1 650 996 3284
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Thomas Spatzier]

Excerpt from Zane Bitter's message on 04/03/2014 23:16:21:
> From: Zane Bitter 
> To: openstack-dev@lists.openstack.org
> Date: 04/03/2014 23:20
> Subject: Re: [openstack-dev] Incubation Request: Murano
>
> On 04/03/14 00:04, Georgy Okrokvertskhov wrote:
> >
> It so happens that the OASIS's TOSCA technical committee are working as
> we speak on a "TOSCA Simple Profile" that will hopefully make things
> easier to use and includes a YAML representation (the latter is great
> IMHO, but the key to being able to do it is the former). Work is still
> at a relatively early stage and in my experience they are very much open
> to input from implementers.

Nice, I was probably also writing a mail with this information at about the
same time :-)
And yes, we are very much interested in feedback from implementers and open
to suggestions. If we can find gaps and fill them with good proposals, now
is the right time.

>
> I would strongly encourage you to get involved in this effort (by
> joining the TOSCA TC), and also to architect Murano in such a way that
> it can accept input in multiple formats (this is something we are making
> good progress toward in Heat). Ideally the DSL format for Murano+Heat
> should be a trivial translation away from the relevant parts of the YAML
> representation of TOSCA Simple Profile.

Right, having a straight-forward translation would be really desirable. The
way to get there can actually be two-fold: (1) any feedback we get from the
Murano folks on the TOSCA simple profile and YAML can help us to make TOSCA
capable of addressing the right use cases, and (2) on the other hand make
sure the implementation goes in a direction that is in line with what TOSCA
YAML will look like.

>
> cheers,
> Zane.
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Zane Bitter]

On 04/03/14 00:04, Georgy Okrokvertskhov wrote:


First of all let me highlight that Murano DSL was much inspired by
TOSCA. We carefully read this standard before our movement to Murano
DSL. TOSCA standard has a lot f of very well designed concepts and ideas
which we reused in Murano. There is one obvious draw back of TOSCA -
very heavy and verbose XML based syntax. Taking into account that
OpenStack itself is clearly moving from XML based representations, it
will be strange to bring this huge XML monster back on a higher level.
Frankly, the current Murano workflows language is XML based and it is
quite painful to write a workflows without any additional instrument
like IDE.


It so happens that the OASIS's TOSCA technical committee are working as 
we speak on a "TOSCA Simple Profile" that will hopefully make things 
easier to use and includes a YAML representation (the latter is great 
IMHO, but the key to being able to do it is the former). Work is still 
at a relatively early stage and in my experience they are very much open 
to input from implementers.


I would strongly encourage you to get involved in this effort (by 
joining the TOSCA TC), and also to architect Murano in such a way that 
it can accept input in multiple formats (this is something we are making 
good progress toward in Heat). Ideally the DSL format for Murano+Heat 
should be a trivial translation away from the relevant parts of the YAML 
representation of TOSCA Simple Profile.


cheers,
Zane.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Thomas Spatzier]

Hi all,

I would like to pick up the TOSCA topic brought up by Zane in his mail
below.

TOSCA is in fact a standard that seems closely aligned with the concepts
that Murano is implementing, so thanks Zane for bringing up this
discussion. I saw Georgy's reply early today where he stated that Murano is
actually heavily inspired by TOSCA, but Murano took a different path due to
some drawbacks in TOSCA v1.0 (e.g. XML).

I would like to point out, though, that we (TOSCA TC) are heavily working
on fixing some of the usability issues that TOSCA v1.0 has. The most
important point being that we are working on a YAML rendering, along with a
simplified profile of TOSCA, which both shall make it easier and more
attractive to use TOSCA. Much of this work has actually been inspired by
the collaboration with the Heat community and the development of the HOT
language.

That said, I would really like the Murano folks to have a look at a current
working draft of the TOSCA Simple Profile in YAML which can be found at
[1]. It would be nice to get some feedback, and ideally we could even
collaborate to see if we can come up with a common solution that fits
everyone's needs. As far as topologies are concerned, we are trying to get
TOSCA YAML and HOT well aligned so we can have an easy mapping. Sahdev from
our team (IRC spzala) is actually working on a TOSCA YAML to HOT converter
which he recently put on stackforge (initial version only). With Murano it
would be interesting to see if we could collaborate on the "plans" side of
TOSCA.

Apart from pure DSL work, I think Murano has some other interesting items
that are also interesting from a TOSCA perspective. For example, I read
about a catalog that stores artifacts needed for app deployments. TOSCA
also has the concept of artifacts, and we have a packaging format to
transport a model and its associated artifacts. So if at some point we
start thinking about importing such a TOSCA archive into a layer above
today's Heat, the question is if we could use e.g. the Murano catalog for
storing all content.

All that said, I see some good opportunities for collaboration and it would
be nice to find a common solution with good alignment between projects and
to avoid duplicate efforts.

BTW, Georgy, I am impressed how closely you looked at the TOSCA spec and
the charter :-)

[1]
https://www.oasis-open.org/committees/document.php?document_id=52381&wg_abbrev=tosca

Greetings,
Thomas

Zane Bitter  wrote on 04/03/2014 03:33:01:

> From: Zane Bitter 
> To: openstack-dev@lists.openstack.org
> Date: 04/03/2014 03:32
> Subject: Re: [openstack-dev] Incubation Request: Murano
>
> On 25/02/14 05:08, Thierry Carrez wrote:
> > The second challenge is that we only started to explore the space of
> > workload lifecycle management, with what looks like slightly
overlapping
> > solutions (Heat, Murano, Solum, and the openstack-compatible PaaS
> > options out there), and it might be difficult, or too early, to pick a
> > winning complementary set.
>
> I'd also like to add that there is already a codified OASIS standard
> (TOSCA) that covers application definition at what appears to be a
> similar level to Murano. Basically it's a more abstract version of what
> Heat does plus workflows for various parts of the lifecycle (e.g.
backup).
>
> Heat and TOSCA folks have been working together since around the time of
> the Havana design summit with the aim of eventually getting a solution
> for launching TOSCA applications on OpenStack. Nothing is set in stone
> yet, but I would like to hear from the Murano folks how they are
> factoring compatibility with existing standards into their plans.
>
> cheers,
> Zane.
>


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Georgy Okrokvertskhov]

Hi,

Here is an etherpad page with current Murano status
http://etherpad.openstack.org/p/murano-incubation-status.

Thanks
Georgy


On Mon, Mar 3, 2014 at 9:04 PM, Georgy Okrokvertskhov <
gokrokvertsk...@mirantis.com> wrote:

> Hi Zane,
>
> Thank you very much for this question.
>
> First of all let me highlight that Murano DSL was much inspired by TOSCA.
> We carefully read this standard before our movement to Murano DSL. TOSCA
> standard has a lot f of very well designed concepts and ideas which we
> reused in Murano. There is one obvious draw back of TOSCA - very heavy and
> verbose XML based syntax. Taking into account that OpenStack itself is
> clearly moving from XML based representations, it will be strange to bring
> this huge XML monster back on a higher level. Frankly, the current Murano
> workflows language is XML based and it is quite painful to write a
> workflows without any additional instrument like IDE.
>
> Now let me remind that TOSCA has a defined scope of its responsibility.
> There is a list of areas which are out of scope. For Murano it was
> important to see that the following items are out of TOSCA scope:
> Citations from [1]:
> ...
> 2. The definition of concrete plans, i.e. the definition of plans in any
> process modeling language like BPMN or BPEL.
> 3. The definition of a language for defining plans (i.e. a new process
> modeling language).
> ...
> Plans in TOSCA understanding is something similar to workflows. This is
> what we address by Murano workflow.
>
> Not let me go through TOSCA ideas and show how they are reflected in
> Murano. It will be a long peace of text so feel free to skip it.
>
> Taking this into account lets review what we have in Murano as an
> application package. Inside application package we have:
> 1. Application metadata which describes application, its relations and
> properties
> 2. Heat templates snippets
> 3. Scripts for deployment
> 4. Workflow definitions
>
> In TOSCA document in section 3.2.1 there are Service Templates introduced.
> These templates are declarative descriptions of services components and
> service Topologies. Service templates can be stored in catalog to be found
> and used by users. This  service template description is abstracted from
> actual infrastructure implementation and each cloud provider maps this
> definition to actual cloud infrastructure. This is definitely a part which
> is already covered by Heat.
> The same section says the following:  "Making a concrete instance of a
> Topology Template can be done by running a corresponding Plan (so-called
> instantiating management plan, a.k.a. build plan). This build plan could be
> provided by the service developer who also creates the Service Template." This
> plan part which is out of scope of TOSCA is essentially what Murano adds as
> a part of application definition.
>
> Section 3.3 of TOSCA document introduces an new entity - artifacts.
> Artifact is a name for content which is needed for service deployment
> including (scripts, executables, binaries and images). That is why Murano
> has a metadata service to store artifacts as a part of application package.
> Moreover, Murano works with Glance team to move this metadata repository
> from Murano to Glance providing generic artifact repository which can be
> used not only by Murano but by any other services.
>
> Sections 3.4 and 3.5 explain the idea of Relationships with
> Compatibilities and Service Composition. Murano actually implements all
> these high level features. Application definition has a section with
> contract definitions. This contract syntax is not just a declaration of the
> relations and capabilities but also a way to make assertions and on-the fly
> type validation and conversion if needed.  Section 10 reveals details of
> requirements. It explains that requirements can be complex: inherit each
> other and be abstract to define a broad set of required values. Like when
> service requires relation database it will require type=RDMS without
> assuming the actual DB implementation MySQL, PostgreSQL or MSSQL.
>
> In order to solve the problem of complex requirements, relations and
> service composition we introduced  classes in our DSL. It was presented and
> discussed in this e-mail thread [3]. Murano DSL syntax allows application
> package writer to compose applications from existing classes by using
> inheritance and class properties with complex types like classes. It is
> possible to define a requirement with using abstract classes to define
> specific types of applications and services like databases, webservers and
> other. Using class inheritance Murano will be able to satisfy the
> requirement by specific object which proper parent class by checking the
> whole hierarchy of objects parent classes which can be abstract.
>
> I don't want to cover all entities defined in TOSCA as most important were
> discussed already. There are implementations of many TOSCA concepts in
> Murano, like class properti

Re: [openstack-dev] Incubation Request: Murano

[Georgy Okrokvertskhov]

Hi Zane,

Thank you very much for this question.

First of all let me highlight that Murano DSL was much inspired by TOSCA.
We carefully read this standard before our movement to Murano DSL. TOSCA
standard has a lot f of very well designed concepts and ideas which we
reused in Murano. There is one obvious draw back of TOSCA - very heavy and
verbose XML based syntax. Taking into account that OpenStack itself is
clearly moving from XML based representations, it will be strange to bring
this huge XML monster back on a higher level. Frankly, the current Murano
workflows language is XML based and it is quite painful to write a
workflows without any additional instrument like IDE.

Now let me remind that TOSCA has a defined scope of its responsibility.
There is a list of areas which are out of scope. For Murano it was
important to see that the following items are out of TOSCA scope:
Citations from [1]:
...
2. The definition of concrete plans, i.e. the definition of plans in any
process modeling language like BPMN or BPEL.
3. The definition of a language for defining plans (i.e. a new process
modeling language).
...
Plans in TOSCA understanding is something similar to workflows. This is
what we address by Murano workflow.

Not let me go through TOSCA ideas and show how they are reflected in
Murano. It will be a long peace of text so feel free to skip it.

Taking this into account lets review what we have in Murano as an
application package. Inside application package we have:
1. Application metadata which describes application, its relations and
properties
2. Heat templates snippets
3. Scripts for deployment
4. Workflow definitions

In TOSCA document in section 3.2.1 there are Service Templates introduced.
These templates are declarative descriptions of services components and
service Topologies. Service templates can be stored in catalog to be found
and used by users. This  service template description is abstracted from
actual infrastructure implementation and each cloud provider maps this
definition to actual cloud infrastructure. This is definitely a part which
is already covered by Heat.
The same section says the following:  "Making a concrete instance of a
Topology Template can be done by running a corresponding Plan (so-called
instantiating management plan, a.k.a. build plan). This build plan could be
provided by the service developer who also creates the Service Template." This
plan part which is out of scope of TOSCA is essentially what Murano adds as
a part of application definition.

Section 3.3 of TOSCA document introduces an new entity - artifacts.
Artifact is a name for content which is needed for service deployment
including (scripts, executables, binaries and images). That is why Murano
has a metadata service to store artifacts as a part of application package.
Moreover, Murano works with Glance team to move this metadata repository
from Murano to Glance providing generic artifact repository which can be
used not only by Murano but by any other services.

Sections 3.4 and 3.5 explain the idea of Relationships with Compatibilities
and Service Composition. Murano actually implements all these high level
features. Application definition has a section with contract definitions.
This contract syntax is not just a declaration of the relations and
capabilities but also a way to make assertions and on-the fly type
validation and conversion if needed.  Section 10 reveals details of
requirements. It explains that requirements can be complex: inherit each
other and be abstract to define a broad set of required values. Like when
service requires relation database it will require type=RDMS without
assuming the actual DB implementation MySQL, PostgreSQL or MSSQL.

In order to solve the problem of complex requirements, relations and
service composition we introduced  classes in our DSL. It was presented and
discussed in this e-mail thread [3]. Murano DSL syntax allows application
package writer to compose applications from existing classes by using
inheritance and class properties with complex types like classes. It is
possible to define a requirement with using abstract classes to define
specific types of applications and services like databases, webservers and
other. Using class inheritance Murano will be able to satisfy the
requirement by specific object which proper parent class by checking the
whole hierarchy of objects parent classes which can be abstract.

I don't want to cover all entities defined in TOSCA as most important were
discussed already. There are implementations of many TOSCA concepts in
Murano, like class properties for TOSCA Properties, class methods for TOSCA
Operations etc.

TL;DR

Summarizing everything I wrote above, I can say that Murano is a
lightweight implementation of TOSCA ideas without XML syntax and
declarative parts which are covered by Heat. This is definitely not an
implementation of TOSCA standard as is, but I am not sure that this is
event necessary. If TOSCA standard will b

Re: [openstack-dev] Incubation Request: Murano

[Zane Bitter]

On 25/02/14 05:08, Thierry Carrez wrote:

The second challenge is that we only started to explore the space of
workload lifecycle management, with what looks like slightly overlapping
solutions (Heat, Murano, Solum, and the openstack-compatible PaaS
options out there), and it might be difficult, or too early, to pick a
winning complementary set.


I'd also like to add that there is already a codified OASIS standard 
(TOSCA) that covers application definition at what appears to be a 
similar level to Murano. Basically it's a more abstract version of what 
Heat does plus workflows for various parts of the lifecycle (e.g. backup).


Heat and TOSCA folks have been working together since around the time of 
the Havana design summit with the aim of eventually getting a solution 
for launching TOSCA applications on OpenStack. Nothing is set in stone 
yet, but I would like to hear from the Murano folks how they are 
factoring compatibility with existing standards into their plans.


cheers,
Zane.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Thierry Carrez]

Georgy Okrokvertskhov wrote:
> [...]
> As you can see this is complicated topic with a number of possible
> solutions. What Murano team is seeking to achieve is to get feedback of
> community and TC on the most appropriate way to structure the governance
> model for the project.

And that should make for an interesting discussion. Like you say it's a
complicated topic, and we need a coherent and integrated solution in the
end. I think there are two unique challenges here.

The first is that Murano looks more like a complete solution (rather
than an infrastructure piece), which makes it span features that we
currently find in separate programs (user-facing catalog/discovery,
workload lifecycle management). My point in my earlier email is that we
could maybe bring that new functionality to OpenStack as a whole without
necessarily introducing a new component, by breaking those features into
the existing projects. Because in the end, OpenStack is not a collection
of overlapping products, it's a set of complementary components.

The second challenge is that we only started to explore the space of
workload lifecycle management, with what looks like slightly overlapping
solutions (Heat, Murano, Solum, and the openstack-compatible PaaS
options out there), and it might be difficult, or too early, to pick a
winning complementary set.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Georgy Okrokvertskhov]

Hi Thierry,


Let me clarify the situation with existing programs and projects overlap.
First of all, I would like to separate questions about what program Murano
as a project can fit and about any overlap with existing projects in the
official programs.


We position Application Catalog as a project that provides functionality
for application publishing, distribution and management. What we were
suggesting is that Murano as a project might fit in a Catalog program which
technically does not exist yet, but this could possibly be one of the ways
how current Image program might evolve.


On the project level, I don't see any overlap with existing Glance project
functionality. The Murano team is actively working with the Glance team to
define a roadmap towards more generic metadata repository functionality for
storing metadata not only images but other artifacts like application
packages, Heat templates etc. Once this roadmap is realized, Murano would
use a Glance repository for storing metadata.


Let me also explain why we listed Orchestration program as a possible place
for Murano. The Orchestration mission states that the goal "is to create a
human- and machine-accessible service for managing the entire lifecycle of
infrastructure and applications within OpenStack clouds". An application
catalog provides self-service capabilities for a cloud user to manage
applications on top of the cloud. In this form, the mission of the
Orchestration program can be applied to the Murano project.


The fact that Murano fits the Orchestration mission does not mean that
there is an overlap with existing projects in this program. Murano uses
Heat to perform actual deployment. In this sense Murano does not deploy
most things directly. Murano uses application definition to generate a Heat
template from Heat template snippets. A good analogy here is the TripleO
project which combines Heat templates based on desired OpenStack
configuration and uses Heat to perform actual work.


The key functionality in Murano is an application package definition. An
application package consists of a UI definition, metadata to control its
appearance in the Catalog, requirements which help Catalog to find a
required or dependent  applications, rules to control Heat template
definitions from snippets and scripts which are part of application
packages too. An essential requirement is to keep Murano project solid as
it covers all aspects of working with application starting from UI
appearance and ending with controlling a heat template-based deployment.


I also wouldn't completely disregard an option to create a new program
combining few projects to cover aspects of application management.


As you can see this is complicated topic with a number of possible
solutions. What Murano team is seeking to achieve is to get feedback of
community and TC on the most appropriate way to structure the governance
model for the project.

Thanks
Georgy


On Mon, Feb 24, 2014 at 2:24 AM, Thierry Carrez 
wrote:
>
> Mark Washenberger wrote:
> > Prior to this email, I was imagining that we would expand the Images
> > program to go beyond storing just block device images, and into more
> > structured items like whole Nova instance templates, Heat templates, and
> > Murano packages. In this scheme, Glance would know everything there is
> > to know about a resource--its type, format, location, size, and
> > relationships to other resources--but it would not know or offer any
> > links for how a resource is to be used.
>
> I'm a bit uncomfortable as well. Part of our role at the Technical
> Committee is to make sure additions do not overlap in scope and make
> sense as a whole.
>
> Murano seems to cover two functions. The first one is publishing,
> cataloging and discovering software stacks. The second one is to deploy
> those software stacks and potentially manage their lifecycle.
>
> In the OpenStack "integrated" release we already have Glance as a
> publication/catalog/discovery component and Heat as the workload
> orchestration end. Georgy clearly identified those two facets, since the
> incubation request lists those two programs as potential homes for Murano.
>
> The problem is, Orchestration doesn't care about the Catalog part of
> Murano, and Glance doesn't care about the Orchestration part of Murano.
> Murano spans the scope of two established programs. It's not different
> enough to really warrant its own program, and it's too monolithic to fit
> in our current landscape.
>
> I see two ways out: Murano can continue to live as a separate
> application that lives on top of OpenStack and consumes various
> OpenStack components. Or its functionality can be split and subsumed by
> Glance and Heat, with Murano developers pushing it there. There seems to
> be interest in both those programs to add features that Murano covers.
> The question is, could we replicate Murano's featureset completely in
> those existing components ? Or is there anything Murano-unique that
> 

Re: [openstack-dev] Incubation Request: Murano

[Jay Pipes]

On Mon, 2014-02-24 at 11:24 +0100, Thierry Carrez wrote:
> Mark Washenberger wrote:
> > Prior to this email, I was imagining that we would expand the Images
> > program to go beyond storing just block device images, and into more
> > structured items like whole Nova instance templates, Heat templates, and
> > Murano packages. In this scheme, Glance would know everything there is
> > to know about a resource--its type, format, location, size, and
> > relationships to other resources--but it would not know or offer any
> > links for how a resource is to be used.
> 
> I'm a bit uncomfortable as well. Part of our role at the Technical
> Committee is to make sure additions do not overlap in scope and make
> sense as a whole.
> 
> Murano seems to cover two functions. The first one is publishing,
> cataloging and discovering software stacks. The second one is to deploy
> those software stacks and potentially manage their lifecycle.
> 
> In the OpenStack "integrated" release we already have Glance as a
> publication/catalog/discovery component and Heat as the workload
> orchestration end. Georgy clearly identified those two facets, since the
> incubation request lists those two programs as potential homes for Murano.
> 
> The problem is, Orchestration doesn't care about the Catalog part of
> Murano, and Glance doesn't care about the Orchestration part of Murano.
> Murano spans the scope of two established programs. It's not different
> enough to really warrant its own program, and it's too monolithic to fit
> in our current landscape.
> 
> I see two ways out: Murano can continue to live as a separate
> application that lives on top of OpenStack and consumes various
> OpenStack components. Or its functionality can be split and subsumed by
> Glance and Heat, with Murano developers pushing it there. There seems to
> be interest in both those programs to add features that Murano covers.

There is a third component: the UI pieces. This naturally would belong
in the UX program.

> The question is, could we replicate Murano's featureset completely in
> those existing components ? Or is there anything Murano-unique that
> wouldn't fit in existing projects ?

Outside of its innovative UX form-construction component, the biggest
thing that makes Murano unique (IMO) is its use of flow control
constructs in its DSL. If I'm not mistaken, the Heat community has made
it clear that they do not intend to introduce flow control constructs
into HOT, and so there would be this piece that would need to live
outside of Heat, but still in the Orchestration program. So, I believe
that there is still a compelling reason for Murano to exist as a
separate project within the Orchestration program, with a mission to
provide a higher level DSL for deployment of complex application
topologies that includes flow control contructs.

And then someone will ask "well, isn't that partly what Solum is
designed for?". And we're back to a similar discussion ;)

Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Thierry Carrez]

Mark Washenberger wrote:
> Prior to this email, I was imagining that we would expand the Images
> program to go beyond storing just block device images, and into more
> structured items like whole Nova instance templates, Heat templates, and
> Murano packages. In this scheme, Glance would know everything there is
> to know about a resource--its type, format, location, size, and
> relationships to other resources--but it would not know or offer any
> links for how a resource is to be used.

I'm a bit uncomfortable as well. Part of our role at the Technical
Committee is to make sure additions do not overlap in scope and make
sense as a whole.

Murano seems to cover two functions. The first one is publishing,
cataloging and discovering software stacks. The second one is to deploy
those software stacks and potentially manage their lifecycle.

In the OpenStack "integrated" release we already have Glance as a
publication/catalog/discovery component and Heat as the workload
orchestration end. Georgy clearly identified those two facets, since the
incubation request lists those two programs as potential homes for Murano.

The problem is, Orchestration doesn't care about the Catalog part of
Murano, and Glance doesn't care about the Orchestration part of Murano.
Murano spans the scope of two established programs. It's not different
enough to really warrant its own program, and it's too monolithic to fit
in our current landscape.

I see two ways out: Murano can continue to live as a separate
application that lives on top of OpenStack and consumes various
OpenStack components. Or its functionality can be split and subsumed by
Glance and Heat, with Murano developers pushing it there. There seems to
be interest in both those programs to add features that Murano covers.
The question is, could we replicate Murano's featureset completely in
those existing components ? Or is there anything Murano-unique that
wouldn't fit in existing projects ?

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Incubation Request: Murano

[Mark Washenberger]

Hi Georgy,

Thanks for all your efforts putting this together.

In the incubation request, one of the proposals is to include Murano under
an expanded scope of the Images program, renaming it the Catalog program.
I've been extremely pleased with the help of you and your colleagues in
helping to define the broader role for Glance as a more general artifact
repository. However, the proposal to bring all of Murano under the expanded
Images program strains my current understanding of how Images needs to
expand in scope.

Prior to this email, I was imagining that we would expand the Images
program to go beyond storing just block device images, and into more
structured items like whole Nova instance templates, Heat templates, and
Murano packages. In this scheme, Glance would know everything there is to
know about a resource--its type, format, location, size, and relationships
to other resources--but it would not know or offer any links for how a
resource is to be used.

For example, Glance would know the virtual size, the storage format, and
all the data associated with a disk image. But it would not necessarily
know anything about a user's ability to either boot that disk image in Nova
or to populate a Cinder volume with the image data.

I think you make a very good point, however. In an orchestrated view of the
cloud, the most usable approach is to have links directly from a resource
to the actions you can perform with the resource. In pseudocode,
image.boot() rather than nova.boot(image). In this more expansive view of
the Catalog, I think it would make sense to include Murano entirely as part
of the Catalog program.

However, this change seems to me to imply a significant architectural shift
for OpenStack in general, and I'm just not quite comfortable endorsing it.
I'm very eager to hear other opinions about this question--perhaps I am
simply not understanding the advantages.

In any case, I hope these notes help to frame the question of where Murano
can best fit.

Thanks again,
markwash


On Thu, Feb 20, 2014 at 10:35 AM, Georgy Okrokvertskhov <
gokrokvertsk...@mirantis.com> wrote:

> All,
>
> Murano is the OpenStack Application Catalog service which has been
> developing on stackforge almost 11 months. Murano has been presented on HK
> summit on unconference track and now we would like to apply for incubation
> during Juno release.
>
> As the first step we would like to get feedback from TC on Murano
> readiness from OpenStack processes standpoint as well as open up
> conversation around mission and how it fits OpenStack ecosystem.
>
> Murano incubation request form is here:
> https://wiki.openstack.org/wiki/Murano/Incubation
>
> As a part of incubation request we are looking for an advice from TC on
> the governance model for Murano. Murano may potentially fit to the
> expanding scope of Image program, if it will be transformed to Catalog
> program. Also it potentially fits Orchestration program, and as a third
> option there might be a value in creation of a new standalone Application
> Catalog program. We have pros and cons analysis in Murano Incubation
> request form.
>
> Murano team  has been working on Murano as a community project. All our
> code and bugs/specs are hosted at OpenStack Gerrit and Launchpad
> correspondingly. Unit tests and all pep8/hacking checks are run at
> OpenStack Jenkins and we have integration tests running at our own Jenkins
> server for each patch set. Murano also has all necessary scripts for
> devstack integration. We have been holding weekly IRC meetings for the last
> 7 months and discussing architectural questions there and in openstack-dev
> mailing lists as well.
>
> Murano related information is here:
>
> Launchpad: https://launchpad.net/murano
>
> Murano Wiki page: https://wiki.openstack.org/wiki/Murano
>
> Murano Documentation: https://wiki.openstack.org/wiki/Murano/Documentation
>
> Murano IRC channel: #murano
>
> With this we would like to start the process of incubation application
> review.
>
> Thanks
> Georgy
>
> --
> Georgy Okrokvertskhov
> Architect,
> OpenStack Platform Products,
> Mirantis
> http://www.mirantis.com
> Tel. +1 650 963 9828
> Mob. +1 650 996 3284
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev