Re: [openstack-dev] [murano] Nominating Felipe Monteiro for murano core
+1 On Thu, Nov 17, 2016 at 4:29 PM Kirill Zaitsev <kzait...@mirantis.com> wrote: > Hello team, I would like to nominate Felipe for murano core. > His and his teams work on increasing murano unit test coverage in Newton > and Ocata has allowed us to reach astonishing 85% > I would like to personally thank Felipe for his dedication to making > murano a more mature and stable project! > You can view statistics for his work here: > http://stackalytics.com/?metric=commits=ocata=murano-group_id=felipe.monte...@att.com > > Please respond with +1/-1 on the candidate =) > > -- > Kirill Zaitsev > Sent with Airmail > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Developer's productivity with MuranoPL
Hi folks,
The developer's productivity and ease of use is critical for the language
adoption, and I believe we have issues with these in MuranoPL. The things
are slowly getting better (thanks to the guys who are bulding th MuranoPL
syntax checker [1] which will be a very useful tool), but that's not
enough: we need more tools to simplify dev's life.
Let me provide a quick example: what does it take to run a "Hello World" in
MuranoPL?
To create such an introductory MuranoPL program the developer has to:
1) Install Murano with the rest of openstack (i.e. spin up devstack with
Murano and Glare plugins)
2) Write the actual HelloWorld "program" (mind the absence of simple
"print" function and the non-so-obvious need to extend
io.murano.Application class for the example to be runnable as murano's
entry point)
3) Write the UI.yaml file to provide the input object graph (even if there
is no user-supplied paramters - we still have to have this file)
4) Write a manifest.yaml
5) zip the program, the ui and the manifest into a package
6) Upload the package to Glare
7) Create an environment in Murano dashboard
8) Add the demo app to the environment
9) Deploy the environment
I hope I din't miss any step. Even excluding the time to spin up devstack
(and the high probability that a newcomer will not do that properly from
the first attempt) this is going to take at least 30 minutes. Even when the
environment is set up, the whole "make code changes - recreate the package
- reupload the package - recreate the environment - redepoy" cycle is
cumbersome.
What do we need is a simple and easy way to run MuranoPL code without the
need to set up all the server-sides components, generate object model and
interact with all the production-grade machinery murano has under the hood.
To do something like:
$ cat "- print('Hello, World')" > ./hello.yaml
$ murano-pl ./hello.yaml
Ideally there should be an interactive REPL-like shell, with smart
indentation and code completion similar to the Python's (we have one for
yaql, and so we should for MuranoPL).
That is not a very hard thing to do, and it will simplify the developers'
onboarding dramatically.
So, I propose to start with a simple thing: to separate the MuranoPL
interpreter (mostly the code located in murano.dsl package, plus some other
stuff) from the rest of Murano. Put it into a standalone reporitory, so it
may be packaged and distributed independently. The developers will just
need to 'pip install murano-pl' to have the local interpreter without all
the dependencies on murano api, engine, database etc. This new package may
include the murano-pl test-runner (this tool is currently part of the main
murano repo and is hard to use since it requires to have a valid config
file which is not a proper option for a cli tool). Then we may include some
other devloper-side tools, such as a murano-pl debugger (when we finally
have one: this is a separate topic).
Finally, we will need to remove the core library (murano.engine.system)
package from the main murano repo and also make it a standalone
repo/package with its own lifecycle (it would be very helpfull if we could
release/update the core library more frequently). So the main murano repo
(and appropriate package) will contain only the server-side murano
components: the REST API, the engine, DB api, model and migrations etc.
When this is done we may begin adding other developer's productivity tools:
starting with debugger and then to various kinds of IDE integrations.
Thoughts, opinions?
[1]
https://github.com/openstack/murano-specs/blob/master/specs/newton/approved/validation-tool.rst
--
Regards,
Alexander Tivelkov
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance][TC][Heat][App-Catalog][Murano][Tacker] Glare as a new Project
I am the one who started the initiative 2.5 years ago, and was always advocating the "let's stay in Glance" approach during numerous discussions on "where should it belong" for all these years. Now I believe that it is time to move forward indeed. Some things remain to be defined (first of all the differences and responsibility sharing between Images and Artifacts APIs), but I am fully supportive of this move and strongly believe it is a step in a right direction. Thanks Mike, Nikhil, Flavio, Erno, Stuart, Brian and all others who helped Glare on this rough path. On Thu, Aug 4, 2016 at 6:29 PM Mikhail Fedosin <mfedo...@mirantis.com> wrote: > Hi all, > after 6 months of Glare v1 API development we have decided to continue our > work in a separate project in the "openstack" namespace with its own core > team (me, Kairat Kushaev, Darja Shkhray and the original creator - > Alexander Tivelkov). We want to thank Glance community for their support > during the incubation period, valuable advice and suggestions - this time > was really productive for us. I believe that this step will allow the Glare > project to concentrate on feature development and move forward faster. > Having the independent service also removes inconsistencies in > understanding what Glance project is: it seems that a single project cannot > own two different APIs with partially overlapping functionality. So with > the separation of Glare into a new project, Glance may continue its work on > the OpenStack Images API, while Glare will become the reference > implementation of the new OpenStack Artifacts API. > > Nevertheless, Glare team would like to continue to collaborate with the > Glance team in a new - cross-project - format. We still have lots in > common, both in code and usage scenarios, so we are looking forward for > fruitful work with the rest of the Glance team. Those of you guys who are > interested in Glare and the future of Artifacts API are also welcome to > join the Glare team: we have a lot of really exciting tasks and will always > welcome new members. > Meanwhile, despite the fact that my focus will be on the new project, I > will continue to be part of the Glance team and for sure I'm going to > contribute in Glance, because I am interested in this project and want to > help it be successful. > > We'll have the formal patches pushed to project-config earlier next week, > appropriate repositories, wiki and launchpad space will be created soon as > well. Our regular weekly IRC meeting remains intact: it is 17:30 UTC > Mondays in #openstack-meeting-alt, it will just become a Glare project > meeting instead of a Glare sub-team meeting. Please feel free to join! > > Best regards, > Mikhail Fedosin > > P.S. For those of you who may be curious on the project name. We'll still > be called "Glare", but since we are on our own now this acronym becomes > recursive: GLARE now stands for "GLare Artifact REpository" :) > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [murano] Plugin installation issue
Hi Vahid, Sorry for the delayed response. I've run your plugin and I am facing an error during the loading of your plugin: the loader complains that it is unable to find the module named "heat_translator". As far as I can see, your plugin attempts the following import statement: *"from heat_translator import translator_shell"* and this is the exact statement which fails. Most probably this happens because the heat_translator project does not expose its heat_translator.py file (the one defined in the root of the module) to the outer world, thus when you install it with pip this module cannot be found. So, if you replace this import statement with something like "*from translator import shell as translator_shell*" this should work, since the "translator" module is properly exported by the heat-translator package. Hope this helps On Mon, Jan 11, 2016 at 10:51 PM Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hello, > > I have been scratching my head at this issue for a couple of days without > any success. > > Here's the issue: > I try to install my plugin (can be found here > <https://review.openstack.org/#/c/243872/>as WIP) using this command: > > pip install -e contrib/plugins/murano_heat-translator_plugin/ > > The installer runs and reports that "Successfully installed > io.murano.plugins.oasis.tosca". > However, when I try to import a package to the catalog I see (while > debugging) that the plugin format I just installed is not among the > plugin_loader formats that murano discovers (here > <https://github.com/openstack/murano/blob/master/murano/packages/load_utils.py#L100> > ). > > I run the same scenario for the Cloudify plugin and everything goes fine. > > So it appears there is something wrong with my package (I was able to > install it before). But I don't know where to look for the root cause of > this issue. > Any pointers would be highly appreciated. > > Thanks. > --Vahid > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [glance][artifacts] FFE for Glare specs
Hi, Unfortunately I skipped the "spec freeze exception week" due to a long holiday season, so I'd like to ask for the freeze exception for the following two specs now: *1. Add more implementation details to 'deprecate-v3-api'* [1] *2. Glare Public API *[2] Spec [1] is actually a patch adding more concrete details to the spec which describes the removal of glance v3 API in favour of standalone glare v0.1 API ([3]), which was accepted for Mitaka and merged. So, it makes no sense to me in accepting [3] but postponing [1] which actually just adds more details of the very same job. The second spec ([2]) aims to stabilise the glare API by addressing DefCore and API-WG comments to the currently present API. The discussions of this API tend to take a lot, but the actual implementation is really quick (since these are just changes in API routers with the same domain and DB and code underneath), and I believe that we will still be able to do this work in Mitaka, even if the spec will be approved much later in the cycle. Also, we've agreed that for this type of work our FastTrack approach should still be applied, which means much less review burden required. Thanks for considering this [1] https://review.openstack.org/#/c/259427/ [2] https://review.openstack.org/#/c/254710/ [3] https://review.openstack.org/#/c/254163/ -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance][artifacts][app-catalog] Proposal to move artifacts meeting time
Hi! This has been implemented: the Artifacts subteam meeting is moved to 17:00 UTC Mondays by patch [1]. However, since we are still deep in the holiday season (and the significant part of the team will be on PTO during the whole next week), I propose to cancel both todays' and the next week's meetings and have the next Glare IRC sync-up on January 11th, 17:00 UTC Have a happy new year! [1] https://review.openstack.org/#/c/260998 On Wed, Dec 23, 2015 at 5:27 PM Alexander Tivelkov <ativel...@mirantis.com> wrote: > Thanks for voting. > > The most popular option is 17:00 UTC Mondays. Unfortunately the > #openstack-meeting-4 channel turned out to be occupied at this timeslot, so > I propose to change the channel to #openstack-meeting-alt > > I've submitted a patch to irc-meeting infra repo: > https://review.openstack.org/#/c/260998 > Please vote for that patch if the channel change is ok to you. > > Thanks! > > On Mon, Dec 21, 2015 at 6:34 AM Nikhil Komawar <nik.koma...@gmail.com> > wrote: > >> Thanks Alex. This is a good idea. Please propose a review for the change >> of schedule so that we can be assured the tests pass and decision would be >> accepted. >> >> >> On 12/18/15 9:20 AM, Alexander Tivelkov wrote: >> >> Hi folks, >> >> The current timeslot of our weekly IRC meeting for artifact subteam >> (14:00 UTC Mondays) seems a bit inconvenient: it's a bit early for people >> in the Pacific timezone. Since we want to maximise the presence of all the >> interested parties at our sync-ups, I'd propose to move our meeting to some >> later timeslot. I'd prefer it to remain in #openstack-meeting-4 (since all >> the rest Glancy meetings are there) and be several days ahead of the main >> Glance meeting (which is on Thursdays). >> >> I've checked the current openstack meetings schedule and found some slots >> which may be more convenient then the current one. I've put them in doodle >> at http://doodle.com/poll/7krdfp96kttnvmg7 - please vote there for the >> slots which are ok for you. Then I'll make a patch to irc-meetings infra >> repo. >> >> Thanks! >> -- >> Regards, >> Alexander Tivelkov >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> -- >> >> Thanks, >> Nikhil >> >> __________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > -- > Regards, > Alexander Tivelkov > -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance][artifacts][app-catalog] Proposal to move artifacts meeting time
Thanks for voting. The most popular option is 17:00 UTC Mondays. Unfortunately the #openstack-meeting-4 channel turned out to be occupied at this timeslot, so I propose to change the channel to #openstack-meeting-alt I've submitted a patch to irc-meeting infra repo: https://review.openstack.org/#/c/260998 Please vote for that patch if the channel change is ok to you. Thanks! On Mon, Dec 21, 2015 at 6:34 AM Nikhil Komawar <nik.koma...@gmail.com> wrote: > Thanks Alex. This is a good idea. Please propose a review for the change > of schedule so that we can be assured the tests pass and decision would be > accepted. > > > On 12/18/15 9:20 AM, Alexander Tivelkov wrote: > > Hi folks, > > The current timeslot of our weekly IRC meeting for artifact subteam (14:00 > UTC Mondays) seems a bit inconvenient: it's a bit early for people in the > Pacific timezone. Since we want to maximise the presence of all the > interested parties at our sync-ups, I'd propose to move our meeting to some > later timeslot. I'd prefer it to remain in #openstack-meeting-4 (since all > the rest Glancy meetings are there) and be several days ahead of the main > Glance meeting (which is on Thursdays). > > I've checked the current openstack meetings schedule and found some slots > which may be more convenient then the current one. I've put them in doodle > at http://doodle.com/poll/7krdfp96kttnvmg7 - please vote there for the > slots which are ok for you. Then I'll make a patch to irc-meetings infra > repo. > > Thanks! > -- > Regards, > Alexander Tivelkov > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > -- > > Thanks, > Nikhil > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [glance][artifacts][app-catalog] Proposal to move artifacts meeting time
Hi folks, The current timeslot of our weekly IRC meeting for artifact subteam (14:00 UTC Mondays) seems a bit inconvenient: it's a bit early for people in the Pacific timezone. Since we want to maximise the presence of all the interested parties at our sync-ups, I'd propose to move our meeting to some later timeslot. I'd prefer it to remain in #openstack-meeting-4 (since all the rest Glancy meetings are there) and be several days ahead of the main Glance meeting (which is on Thursdays). I've checked the current openstack meetings schedule and found some slots which may be more convenient then the current one. I've put them in doodle at http://doodle.com/poll/7krdfp96kttnvmg7 - please vote there for the slots which are ok for you. Then I'll make a patch to irc-meetings infra repo. Thanks! -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance][keystone][artifacts] Service Catalog name for Glance Artifact Repository API
Hi Steve, Thanks for the note on port. Any objections on glare using 9494 then? Anyone? Пт, 11 дек. 2015 г. в 21:39, McLellan, Steven <steve.mclel...@hpe.com>: > Hi Alex, > > Searchlight uses port 9393 (it also made sense to us when we spun out of > Glance!), so we would prefer it if there's another one that makes sense. > Regarding the three hardest things in computer science, searchlight's > already dealing with cache invalidation so I'll stay out of the naming > discussion. > > Thanks! > > Steve > > From: Alexander Tivelkov <ativel...@mirantis.com ativel...@mirantis.com>> > Reply-To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org > >> > Date: Friday, December 11, 2015 at 11:25 AM > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org > >> > Subject: [openstack-dev] [glance][keystone][artifacts] Service Catalog > name for Glance Artifact Repository API > > Hi folks! > > As it was decided during the Mitaka design summit, we are separating the > experimental Artifact Repository API from the main Glance API. This API > will have a versioning sequence independent from the main Glance API and > will be run as a standalone optional service, listening on the port > different from the standard glance-api port (currently the proposed default > is 9393). Meanwhile, it will remain an integral part of the larger Glance > project, sharing the database, implementation roadmap, development and > review teams etc. > > Since this API will be consumed by both end-users and other Openstack > services, its endpoint should be discoverable via regular service catalog > API. This rises the question: what should be the service name and service > type for the appropriate entree in the service catalog? > > We've came out with the idea to call the service "glare" (this is our > internal codename for the artifacts initiative, being an acronym for > "GLance Artifact REpository") and set its type to "artifacts". Other > alternatives for the name may be "arti" or "glance_artifacts" and for the > type - "assets" or "objects" (the latter may be confusing since swift's > type is object-store, so I personally don't like it). > > Well... we all know, naming is complicated... anyway, I'll appreciate any > feedback on this. Thanks! > > -- > Regards, > Alexander Tivelkov > > ______ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [glance][keystone][artifacts] Service Catalog name for Glance Artifact Repository API
Hi folks! As it was decided during the Mitaka design summit, we are separating the experimental Artifact Repository API from the main Glance API. This API will have a versioning sequence independent from the main Glance API and will be run as a standalone optional service, listening on the port different from the standard glance-api port (currently the proposed default is 9393). Meanwhile, it will remain an integral part of the larger Glance project, sharing the database, implementation roadmap, development and review teams etc. Since this API will be consumed by both end-users and other Openstack services, its endpoint should be discoverable via regular service catalog API. This rises the question: what should be the service name and service type for the appropriate entree in the service catalog? We've came out with the idea to call the service "glare" (this is our internal codename for the artifacts initiative, being an acronym for "GLance Artifact REpository") and set its type to "artifacts". Other alternatives for the name may be "arti" or "glance_artifacts" and for the type - "assets" or "objects" (the latter may be confusing since swift's type is object-store, so I personally don't like it). Well... we all know, naming is complicated... anyway, I'll appreciate any feedback on this. Thanks! -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [murano]How to use Murano to transmit files to Mistral and execute scripts on Mistral
Hi Tony, Probably I am missing something, but why do you need Murano Agent to interact with Mistral? You can call Mistral APIs right from MuranoPL code being executed by Murano Engine. Murano's core library contains the io.murano.system.MistralClient class ([1]) which may be used to upload and run mistral workflows. Please let me know if you need more details on this [1] https://github.com/openstack/murano/blob/master/murano/engine/system/mistralclient.py On Tue, Nov 17, 2015 at 5:07 AM WANG, Ming Hao (Tony T) < tony.a.w...@alcatel-lucent.com> wrote: > Dear Murano developers and testers, > > > > I want to put some files that Mistral workflow needs into Murano package, > and hope Murano can transmit them to Mistral before it calls Mistral > workflow. > > The flow should be as following: > > 1. User uploads one Murano package which includes both > Murano artifacts and Mistral artifacts to Murano; > > 2. Murano transmits the Mistral artifacts to Mistral, > and Mistral does its work. > > > > After study, I thought muranoagent may be helpful and plan to install a > muranoagent on the Mistral server since it can put files into nova server, > and can run scripts on the nova server. > > After further study, I found muranoagent solution may be not feasible: > > 1. muranoagent and murano-engine(dsl) uses rabbitMQ to > communicate. > > 2. When an Agent object is created in DSL, > murano-engine creates a unique message queue to communicate with the > muranoagent in nova instance: > > The queue name consists of current murano environment id, and the nova > instance murano object id. > > 3. During murano creates the nova instance, it passes > this unique queue name via nova user_data to muranoagent on guest. > > In this way, muranoagents on different guests can communicate with > murano-engine separately. > > This doesn’t suit the muranoagent + Mistral server solution. > > We only want to install one muranoagent in Mistral server, and it should > only listen on one message queue. > > We can’t create new muranoagent for each murano environment. > > To achieve this, one solution that I can think is to modify murano code to > implement a new MistralAgent to listen on a pre-defined message queue. > > > > Could you please share your ideas about this? > > If you have other solution, please also help to share it. J > > > > Thanks in advance, > > Tony > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [app-catalog] [glance] [murano] Data Assets API for App Catalog
Hi folks, Thanks for the fruitful discussion on the summit: seems like we are in agreement and have a good plan to proceed. Here is the video recording of the PoC I've made to serve the current AppCatalog's data using Glance Artifacts. As the demo was recorded before the summit, it's called Glance V3 everywhere. Now we've agreed that this will be a Glance Artifacts Repository (aka Glare) API v1 - but everything else about its functionality is still valid :) https://youtu.be/5IxKqJiD2xw Please feel free to ask any questions you may have on the demo and Glare functionality in general. Thanks again On Wed, Oct 28, 2015 at 11:38 AM Flavio Percoco <fla...@redhat.com> wrote: > On 23/10/15 19:25 +, Fox, Kevin M wrote: > >The "Glance: Artefacts API" session was scheduled right on top of the > "Nova: > >Cross Service Issues: Service Lock Server, Service Tokens, Instance Users > (TBC) > >" session. The latter having been really hard to schedule since it > involves so > >many different projects and something I must attend. So I won't be able to > >attend the glance session. > > > >The Murano folks have kindly offered to use one of their sessions: > >Murano contributors meetup (9:00am-12:30pm) > > > http://mitakadesignsummit.sched.org/event/5e244fb7f342854dc5c112e76e77c930 > >to discuss Glance Artefacts/Murano integration/Community App Catalog > needs. > > > >Can folks from the glance team that are interested in the discussion > please > >attend? > > Kevin, thanks for the heads up! I'll help spreading the word in case > some folks missed this email. > > Flavio > > > > >Thanks, > >Kevin > > > > >━━━ > >From: Alexander Tivelkov [ativel...@mirantis.com] > >Sent: Thursday, October 15, 2015 3:04 AM > >To: OpenStack Development Mailing List (not for usage questions) > >Subject: [openstack-dev] [app-catalog] [glance] [murano] Data Assets API > for > >App Catalog > > > >Hi folks, > > > >I’ve noticed that the Community Application Catalog has begun to > implement its > >own API, and it seems to me that we are going to have some significant > >duplication of efforts with the code which has already been implemented in > >Glance as Artifact Repository initiative (also known as Glance V3). > >From the very beginning of the App Catalog project (and I’ve been > involved in > >it since February) I’ve been proposing to use Glance as its backend, > because > >from my point of view it covers like 90% of the needed functionality. But > it > >looks like we have some kind of miscommunication here, as I am getting > some > >confusing questions and assumptions, like the vision of Glance V3 being > >dedicated backend for Murano (which is definitely incorrect). > >So, I am writing the email to clarify my vision of what Glance V3 is and > how > >its features may be used to provide the REST API for Community App > Catalog. > > > >1. Versioned schema > >First of all, Glance V3 operates on entities called “artifacts”, and > these ones > >perfectly map to the Data Assets of community app catalog. The artifacts > are > >strongly typed: there are artifact types for murano packages, glance > images, > >heat templates - and there may be (and will be) more. Each artifact type > is > >represented by a plugin, defining the schema of objects’ data and > metadata and > >- optionally - custom logic. So, this thing is extensible: when a new > type of > >asset needs to be added to a catalog it can be done really quickly by just > >defining the schema and putting that schema into a plugin. Also, these > plugins > >are versioned, so the possible changes in the schema are handled properly. > > > >2. Generic properties > >Next, all the artifact types in Glance V3 have some generic metadata > properties > >(i.e. part of the schema which is common for all the types), including the > >name, the version, description, authorship information and so on. This > also > >corresponds to the data schema of community app catalog. The mapping is > not > >1:1, but we can work together on this to make sure that these generic > >properties match the expectations of the catalog. > > > >3. Versioning > >Versions are very important for catalogs of objects, so Glance V3 was > initially > >designed keeping versioning questions in mind: each artifact has a > semver-based > >version assigned, so the artifacts having the same name but different > versions > >are grouped into the proper sequenc
Re: [openstack-dev] [app-catalog] [glance] [murano] Data Assets API for App Catalog
Hi folks! Thanks all for the feedback. Yeah, this definitely needs more discussions, and the Summit is good place to have them. Thanks to Serg for offering a spot in the schedule. I hope to have some PoC by that time, so we can have something which can be demoed. I'll try to attend the AppCatalog sessions as well. I didn't finalize my summit schedule yet, but I'll do my best to attend as much as I can. Thanks! -- Regards, Alexander Tivelkov On Fri, Oct 16, 2015 at 10:07 PM, Christopher Aedo <d...@aedo.net> wrote: > On Thu, Oct 15, 2015 at 3:04 AM, Alexander Tivelkov > <ativel...@mirantis.com> wrote: > > Hi folks, > > > > I’ve noticed that the Community Application Catalog has begun to > implement > > its own API, and it seems to me that we are going to have some > significant > > duplication of efforts with the code which has already been implemented > in > > Glance as Artifact Repository initiative (also known as Glance V3). > > From the very beginning of the App Catalog project (and I’ve been > involved > > in it since February) I’ve been proposing to use Glance as its backend, > > because from my point of view it covers like 90% of the needed > > functionality. But it looks like we have some kind of miscommunication > here, > > as I am getting some confusing questions and assumptions, like the > vision of > > Glance V3 being dedicated backend for Murano (which is definitely > > incorrect). > > So, I am writing the email to clarify my vision of what Glance V3 is and > how > > its features may be used to provide the REST API for Community App > Catalog. > > > > 1. Versioned schema > > First of all, Glance V3 operates on entities called “artifacts”, and > these > > ones perfectly map to the Data Assets of community app catalog. The > > artifacts are strongly typed: there are artifact types for murano > packages, > > glance images, heat templates - and there may be (and will be) more. Each > > artifact type is represented by a plugin, defining the schema of objects’ > > data and metadata and - optionally - custom logic. So, this thing is > > extensible: when a new type of asset needs to be added to a catalog it > can > > be done really quickly by just defining the schema and putting that > schema > > into a plugin. Also, these plugins are versioned, so the possible > changes in > > the schema are handled properly. > > > > 2. Generic properties > > Next, all the artifact types in Glance V3 have some generic metadata > > properties (i.e. part of the schema which is common for all the types), > > including the name, the version, description, authorship information and > so > > on. This also corresponds to the data schema of community app catalog. > The > > mapping is not 1:1, but we can work together on this to make sure that > these > > generic properties match the expectations of the catalog. > > > > 3. Versioning > > Versions are very important for catalogs of objects, so Glance V3 was > > initially designed keeping versioning questions in mind: each artifact > has a > > semver-based version assigned, so the artifacts having the same name but > > different versions are grouped into the proper sequences. API is able to > > query artifacts based on their version spec, e.g. it is possible to fetch > > the latest artifact with the name “foo” having the version greater than > 2.1 > > and less than 3.5.7 - or any other version spec, similar to pip or any > other > > similar tool. As far as I know, community app catalog does not have such > > capabilities right now - and I strongly believe that it is really a must > > have feature for a catalog to be successful. At least it is absolutely > > mandatory for Murano packages, which are the only “real apps” among the > > asset types right now. > > > > 4. Cross artifact dependencies > > Glance V3 also has the dependency relations from the very beginning, so > they > > may be defined as part of artifact type schema. As a result, an artifact > may > > “reference” any number of other artifacts with various semantic. For > > example, murano package may define a set of references to other murano > > packages and call it “requires” - and this will act similar to the > > requirements of a python package. Similar properties may be defined for > heat > > templates and glance images - they may reference each other with various > > semantics. > > Of course, the definitions of such dependencies may be done internally > > inside the packages, so they may be resolved locally by the service > which is > > going to use it, bu
[openstack-dev] [app-catalog] [glance] [murano] Data Assets API for App Catalog
nts) or directly shared by the owner to a specific tenant. Also, Glance can act in the anonymous mode (i.e. without an access token), thus providing access to public artifacts even to unauthenticated users. This can be easily applied to a public web service, such as community app catalog: regular unauthenticated users use anonymous mode to access only the public assets (this is the current behavior of apps.o.o), while registered users will have their own private spaces (“tenants”) with various access restrictions. 8. The federation. The ultimate goal for Glance Artifact Repository is ability to build trees of artifact repos in different clouds. The top node of that tree is some Global Application Catalog (and the apps.openstack.org may be this global catalog - if it is glance-based or at least supports glance v3 federation), then there are repositories of particular openstack vendors or distributors, then - the catalogs of enterprises operating different openstack clouds. The particular glance deployments in that clouds are the leafs of that tree, being able to search for data assets in all the upstream repositories, download them from there or - if permitted - submit their local assets back upstream. This will be the ultimate network for application delivery and exchange in openstack world - and this is one of the main reasons we’ve began the Artifacts initiative in Glance. Unlike other aforementioned features this one is not implemented yet, but we are planning to add it as soon as we are done with API stabilization goal. There are many other features which are present in V3’s roadmap and may be useful for the app catalog, such as ability to sign artifacts with their developers’ keys and verify that keys on usage to ensure the authenticity of the artifact. What we don’t have right now is the ability to associate ratings (“stars”) and comments to the artifact, as well as aggregating different usage and download statistics: such features are really needed only for the public website such as apps.o.o but are not required for Glance’s in particular clouds. But we may find some way to solve this, either by wrapping glance API with additional middleware which would add appropriate info from a different data source, or by having custom plugins which are able to do that, or in some other way: I am sure we may find a solution for this. So, this was just a brief description of what Glance v3 has to offer as a backend for App Catalog API. It also worths to mention that this API is in “EXPERIMENTAL” state right now, which means that it is not fixed and we may modify it significantly if there is a need to. So we may work closer together to adopt it for the needs of Community App Catalog. I would really prefer to not create any overlaps between Glance v3 and the community app catalog: if the app catalog builds its own incompatible implementation of assets discovery and distribution API then we’ll have a huge duplication of efforts for developers and lots of confusion to the end-users who will get two entirely different ways to do the same task. So, I’d propose to discuss these potential overlaps, look at the features need by App Catalog and see how Glance V3 may be of use here. I’ll be more than happy to help with that. We can dive deeper into the details here in the mailing list or meet in person in Tokyo. I'll try to have a demonstratable prototype by that time. -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [murano] [dashboard] Remove the owner filter from "Package Definitions" page
+1 on this. Filtering by ownership makes sense only on Catalog view (i.e. on the page of usable apps) but not on the admin-like console like the list of package definitions. -- Regards, Alexander Tivelkov On Fri, Sep 4, 2015 at 12:36 PM, Dmitro Dovbii <ddov...@mirantis.com> wrote: > Hi folks! > > I want suggest you to delete owner filter (3 tabs) from Package Definition > page. Previously this filter was available for all users and we agreed that > it is useless. Now it is available only for admin but I think this fact > still doesn't improve the UX. Moreover, this filter prevents the > implementation of the search by name, because the work of the two filters > can be inconsistent. > So, please express your opinion on this issue. If you agree, I will remove > this filter ASAP. > > Best regards, > Dmytro Dovbii > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [murano] Proposing Nikolai Starodubtsev for core
+1. Well deserved. -- Regards, Alexander Tivelkov On Tue, Sep 1, 2015 at 2:47 PM, Victor Ryzhenkin <vryzhen...@mirantis.com> wrote: > +1 from me ;) > > -- > Victor Ryzhenkin > Junior QA Engeneer > freerunner on #freenode > > Включено 1 сентября 2015 г. в 12:18:19, Ekaterina Chernova ( > efedor...@mirantis.com) написал: > > +1 > > On Tue, Sep 1, 2015 at 10:03 AM, Dmitro Dovbii <ddov...@mirantis.com> > wrote: > >> +1 >> >> 2015-09-01 2:24 GMT+03:00 Serg Melikyan <smelik...@mirantis.com>: >> >>> +1 >>> >>> On Mon, Aug 31, 2015 at 3:45 PM, Kirill Zaitsev <kzait...@mirantis.com> >>> wrote: >>> >>>> I’m pleased to nominate Nikolai for Murano core. >>>> >>>> He’s been actively participating in development of murano during >>>> liberty and is among top5 contributors during last 90 days. He’s also >>>> leading the CloudFoundry integration initiative. >>>> >>>> Here are some useful links: >>>> >>>> Overall contribution: http://stackalytics.com/?user_id=starodubcevna >>>> List of reviews: >>>> https://review.openstack.org/#/q/reviewer:%22Nikolay+Starodubtsev%22,n,z >>>> Murano contribution during latest 90 days >>>> http://stackalytics.com/report/contribution/murano/90 >>>> >>>> Please vote with +1/-1 for approval/objections >>>> >>>> -- >>>> Kirill Zaitsev >>>> Murano team >>>> Software Engineer >>>> Mirantis, Inc >>>> >>>> >>>> __ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> >>> -- >>> Serg Melikyan, Senior Software Engineer at Mirantis, Inc. >>> http://mirantis.com | smelik...@mirantis.com >>> >>> +7 (495) 640-4904, 0261 >>> +7 (903) 156-0836 >>> >>> >>> __ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [glance][murano] Glance V3 (Artifacts) usage in Murano
Hi folks, In the upcoming L release Murano is going to use the Glance Artifact Repository feature implemented as part of EXPERIMENTAL Glance V3 API. The server-side support of this feature is already merged in glance's master branch, while the client code is not: it was agreed that the v3's client will stay in a dedicated feature branch (feature/artifacts) and will not be released until the API is stable and final (a major API refactoring based on the feedback from API WG is on the way and is likely to happen in M). So, there will be no v3-aware releases of python-glanceclient on pypi until then. The early adopters of V3 API are encouraged to build the tarballs out of the feature branch on their own and use them, keeping in mind that the API is EXPERIMENTAL so everything may be (and will be) changed. However, Murano needs some way to consume Glance V3 right now, and - as it has a voting requirements job at the gate - it cannot just put a git branch reference in its requirements.txt. It needs some kind of a release which would be part of global requirements etc. So, it was decided to temporary copy-paste the experimental part of glance client into the murano client (i.e. to copy python-glanceclient/feature/artifacts/v3 to python-muranoclient/master/artifacts) and release it as part of several next releases of python-muranoclient. When the Glance V3 API is stable, we'll put its client to the master branch of python-glanceclient and release it normally, then dropping the temporary copy from python-muranoclient. Until then, all the changes to the experimental client should be done in the feature/artifacts branch of glance client and copied (synced) to murano client. Similar to oslo.incubator sync procedure, just without a shell script :) I hope that the need to do this copy-pasting will not last for long and the v3 API becomes stable soon enough. -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Does murano dymamic-ui have plan to support edit function?
Hi Tony, Thanks for your interest! This is a complicated topic. Being able to edit the object model (with DynamicUI, the CLI tools mentioned by Kirill or manually with murano's API) is just the tip of the iceberg: if you have already deployed your application, modifying of some of its input properties will not be enough: the application developer has to supply the logic which will handle the changes and will execute all the needed actions to reconfigure the app. Right now the right way to do so is to create an action method (see [1] for details), which may be called for already deployed apps. In this action you may change the properties of the object and do whatever custom handling you may need to reconfigure your app. For example, if you want to change the password of the database admin user of the mysql app, you may add an action method changeAdminPassword which will not only set '$.password' property to a new value, but will also execute an appropriate password-changing script on the VM running the database instance. Right now the actions are partially supported on the UI level (in the dashboard you may call any action of any deployed application), however currently you cannot pass any parameters to these actions if called from the UI, and being able to pass them is indeed required for your scenario (in the aforementioned example with DB password change, such an action should have at least one parameter - the new password value). This will be probably addressed during the M cycle as part of the per-component UI initiative mentioned by Kirill: we will provide a way to render dynamic UI dialogs not only for the new applications being added but also for the actions of the already deployed apps. Hope this helps. Please let me know if you have any questions on Actions and any other related topics [1] http://murano.readthedocs.org/en/latest/draft/appdev-guide/murano_pl.html#murano-actions -- Regards, Alexander Tivelkov On Wed, Aug 12, 2015 at 2:16 PM, WANG, Ming Hao (Tony T) tony.a.w...@alcatel-lucent.com wrote: Kirll, Thanks for your info very much! We will study it first. Thanks, Tony From: Kirill Zaitsev [mailto:kzait...@mirantis.com] Sent: Wednesday, August 12, 2015 7:12 PM To: WANG, Ming Hao (Tony T); OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] Does murano dymamic-ui have plan to support edit function? Hi, sure there are such plans! This have been long referred as per-component-UI. I’m really hoping there would be some traction about it during mitaka cycle. Not in liberty though, feature freeze is less than a month away. btw, if you’re interested in custom tweaking and fine-tuning of murano object-model you can take a look at these CLI tools https://review.openstack.org/#/q/project:openstack/python-muranoclient+branch:master+topic:bp/env-configuration-from-cli,n,z and this https://review.openstack.org/#/c/208659/ commit in particular. Although using those would require you to have some knowledge about how murano handles things internally. -- Kirill Zaitsev Murano team Software Engineer Mirantis, Inc On 12 Aug 2015 at 13:23:47, WANG, Ming Hao (Tony T) (tony.a.w...@alcatel-lucent.com) wrote: Dear OpenStack developers, Currently, murano dynamic-ui is “one-time” GUI, and I can’t edit data what has been submitted. Does murano dynamic-ui have plan to support edit function in the future? For example, developer develops some Wizard GUI to do some configuration, and user wants to change some configuration after the deployment. Thanks, Tony __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [murano] [mistral] [yaql] Prepare to Yaql 1.0 release
Hi folks,
We are finally ready to release the 1.0.0 version of YAQL. It is a
huge milestone: the language finally looks the way we initially wanted
it to look. The engine got completely rewritten, tons of new
capabilities have been added. Here is a brief (and incomplete) list of
new features and improvements:
* Support for kwargs and keyword-only args (Py3)
* Optional function arguments
* Smart algorithm to find matching function overload without side effects
* Ability to organize functions into layers
* Configurable list of operators (left/right associative binary,
prefix/suffix unary with precedence)
* No global variables. There can be more than one parser with
different set of operators simultaneously
* List literals ([a, b])
* Dictionary literals ({ a = b})
* Handling of escape characters in string literals
* Verbatim strings (`...`) and double-quotes (...)
* =~ and !~ operators in default configuration (similar to Perl)
* - operator to pass context
* Alternate operator names (for example '*equal' instead of '#operator_=')
so that it will be possible to have different symbol for particular operator
without breaking standard library that expects operator to have well
known names
* Set operations
* Support for lists and dictionaries as a dictionary keys and set elements
* New framework to decorate functions
* Ability to distinguish between functions and methods
* Switchable naming conventions
* Unicode support
* Execution options available to all invoked functions
* Iterators limitation
* Ability to limit memory consumption
* Can work with custom context classes
* It is possible to extend both parser and set of expression classes
on user-side
* It is possible to create user-defined types (also can be used for
dependency injection)
* Legacy yaql 0.2.x backward compatibility mode
* Comprehensive standard library of functions
* High unit test coverage
* Delegate and lambda support, including higher order lambdas
etc, etc.
So, this is a big change.
And as it always happens when moving from 0.something to 1.x the
breaking changes are inevitable. We have included the backwards
compatibility mode, but it may not address all the possible concerns.
So.
We have released a release candidate 1 of yaql 1.0.0 on pypi: [1]
It includes all the new functionality and is likely to be identical to
final release (that's why it is RC after all) and we strongly
encourage all the yaql users (Murano and Mistral first of all) to try
it and prepare migration patches to use it. When the final release
is out, we'll update the global requirements to yaql = 1.0.0, which
is likely to break all your gate checks unless you quickly land a
migrating patch.
Please email us any concerns or contact me (ativelkov) or Stan Lagun
(slagun) directly in IRC (#murano) if you need some quick help on yaql
1.0 or migrating from 0.2.x
Happy yaqling!
[1] https://pypi.python.org/pypi/yaql/1.0.0.0rc1
--
Regards,
Alexander Tivelkov
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Should we move to #openstack-murano ?
I'd prefer not to multiply entities beyond necessity. This is just a channel name - and a known name already. We can't just rename it, we'll have to maintain both channels for some (significant) amount of time. And this is usually bad: when you remember that there was some discussion and want to find the logs, you'll have to look at both locations instead of a single one. This is usually a headache. So, -1 from my side unless the pattern #openstack-projectname becomes absolutely mandatory. -- Regards, Alexander Tivelkov On Sat, Jul 18, 2015 at 2:05 AM, Kirill Zaitsev kzait...@mirantis.com wrote: Heat does seem to use #heat though. But I have to agree, that most os OpenStack projects use #openstack-something I do not have a strong opinion about the idea, but I’m pretty sure nobody came to #openstack-murano and asked anything there. People usually do not ask questions in empty channels ;))) -- Kirill Zaitsev Murano team Software Engineer Mirantis, Inc On 17 Jul 2015 at 19:23:30, Ekaterina Chernova (efedor...@mirantis.com) wrote: Hi guys! Currently Murano holds the *#murano* IRC channel. But all openstack projects have the openstack- prefix in their channel’s name. So I have a question: should we move to *#openstack-murano*? I think it’s a good idea, since it’s more obvious to go to *#openstack-murano* if one needs help with murano. Do you know if anybody tried to get help at *#openstack-murano* and discovered that this is not the official Murano channel ? Would it be hard to migrate from one channel to another? Regards, Kate. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Versioning of Murano packages and MuranoPL classes
Gosha,
Could you please elaborate what do you mean by extra blocks? Glance V3
comes with Glance out-of-the box, no extra deployment is needed. The only
thing one will have to install is Murano Package Type plugin - but it will
be installed at the same time with Murano.
--
Regards,
Alexander Tivelkov
On Mon, Jul 13, 2015 at 5:54 PM, Georgy Okrokvertskhov
gokrokvertsk...@mirantis.com wrote:
On Mon, Jul 13, 2015 at 2:19 AM, Alexander Tivelkov
ativel...@mirantis.com wrote:
Hi Gosha,
Supporting versioning in existing backend will require us to re-implement
the significant part of Artifact Repository in Murano API: we'll need to
add versions and dependencies concepts into our model (which is already
complicated and dirty enough), extend appropriate API calls etc. And all
the efforts will be a waste of time once we finally migrate to Artifacts.
Also, what do you mean by set by default? V3 API is experimental, but
it is already merged into upstream Glance, so there is no problem with
using it in Murano right away.
This is exactly why I have these concerns. I wonder how much customers
will use experimental API in production. I just don't want to add extra
block on Murano adoption way.
--
Regards,
Alexander Tivelkov
On Fri, Jul 10, 2015 at 5:56 PM, Georgy Okrokvertskhov
gokrokvertsk...@mirantis.com wrote:
Hi Alex,
Thank you for the great summary.
I have a concern about item #8. Can we add an option to Murano to use
previous storage engine rather then Glance v3? We need to make sure that v3
API in Glance is set by default before we do a hard dependency on it in
Murano.
Thanks
Gosha
On Fri, Jul 10, 2015 at 4:53 AM, Alexander Tivelkov
ativel...@mirantis.com wrote:
Hi folks,
Ability to manage multiple versions of application packages and their
dependencies was always an important item in Murano roadmap, however we
still don't have a clear spec for this feature.
Yesterday we hosted a small design session to come up with a plan on
what can be done in Liberty timeframe to have proper versioning for
MuranoPL classes and packages. Stan Lagun, Kirill Zaitsev and myself
participated offline, some other muranoers joined remotely. Thanks to
everybody who joined us.
TL;DR: it turns out that now we have a clear plan which will allow us
to achieve proper versioning of the packages and classes, and we'll try to
implement the most important parts of it in Liberty.
Here is the detailed outcome of the session:
1. We'll use the standard Semantic Versioning format
('Major.Minor.Patch[-dev-build.label[+metadata.label]]') to version our
packages: changes which break backwards-compatibility should increment
the
major segment, non-breaking new features increment the minor segment and
all non-breaking bugfixes increment the patch segment. The developers
should be carefull with the new features part: if you add a new
method to
a class, it may be considered a breaking change if the existing
subclasses
of this class have the same method already declared. We still assume
that
such changes should lead to increment of 'minor' segment, however it is
up
to best judgement of developers in particular case: if the risk of such
method override is really high it may worth to increment the 'major'
segment. Proper guideline on the versioning rules will be published
closer
to L release.
2. A new field 'Version' is introduced into package manifest file
which should define package version in complete semver format. The field
itself is optional (so existing apps are not broken), if it is not
specified the package is assumed to have version '0.0.0'
3. The existing 'Require' block of Application manifest will be
used to specify the package dependencies. Currently it is a yaml-based
dictionary, with the keys set to fully-qualified names of the dependency
packages and the values set to the version of those dependencies.
Currently
this block is used only for integration with apps stored at
apps.openstack.org. It is suggested to use this block in the
deployment process as well, and extend its semantics.
The version of the dependency specified there should also follow
the semver notation, however it may be specified in the shortened
format,
i.e. without specifying the 'patch' or 'patch' and 'minior' components.
In
this case the dependency will be specified as a range of allowed
versions.
For example, a dependency version 1.2 will mean a (1.2.0 = version
1.3)
range.
If the version of a dependency is not specified (like in this
existing app - [1]) then we assume the version 0 - i.e. the last
available pre-release version of a package.
4. Murano core library is also a package which has its own version.
The current one is assumed to have a version 0.1.0, the one which is
going
to be released in L will be probably called 0.2.0
Re: [openstack-dev] [Murano] Versioning of Murano packages and MuranoPL classes
Hi Gosha,
Supporting versioning in existing backend will require us to re-implement
the significant part of Artifact Repository in Murano API: we'll need to
add versions and dependencies concepts into our model (which is already
complicated and dirty enough), extend appropriate API calls etc. And all
the efforts will be a waste of time once we finally migrate to Artifacts.
Also, what do you mean by set by default? V3 API is experimental, but it
is already merged into upstream Glance, so there is no problem with using
it in Murano right away.
--
Regards,
Alexander Tivelkov
On Fri, Jul 10, 2015 at 5:56 PM, Georgy Okrokvertskhov
gokrokvertsk...@mirantis.com wrote:
Hi Alex,
Thank you for the great summary.
I have a concern about item #8. Can we add an option to Murano to use
previous storage engine rather then Glance v3? We need to make sure that v3
API in Glance is set by default before we do a hard dependency on it in
Murano.
Thanks
Gosha
On Fri, Jul 10, 2015 at 4:53 AM, Alexander Tivelkov
ativel...@mirantis.com wrote:
Hi folks,
Ability to manage multiple versions of application packages and their
dependencies was always an important item in Murano roadmap, however we
still don't have a clear spec for this feature.
Yesterday we hosted a small design session to come up with a plan on what
can be done in Liberty timeframe to have proper versioning for MuranoPL
classes and packages. Stan Lagun, Kirill Zaitsev and myself participated
offline, some other muranoers joined remotely. Thanks to everybody who
joined us.
TL;DR: it turns out that now we have a clear plan which will allow us to
achieve proper versioning of the packages and classes, and we'll try to
implement the most important parts of it in Liberty.
Here is the detailed outcome of the session:
1. We'll use the standard Semantic Versioning format
('Major.Minor.Patch[-dev-build.label[+metadata.label]]') to version our
packages: changes which break backwards-compatibility should increment the
major segment, non-breaking new features increment the minor segment and
all non-breaking bugfixes increment the patch segment. The developers
should be carefull with the new features part: if you add a new method
to
a class, it may be considered a breaking change if the existing subclasses
of this class have the same method already declared. We still assume that
such changes should lead to increment of 'minor' segment, however it is up
to best judgement of developers in particular case: if the risk of such
method override is really high it may worth to increment the 'major'
segment. Proper guideline on the versioning rules will be published closer
to L release.
2. A new field 'Version' is introduced into package manifest file
which should define package version in complete semver format. The field
itself is optional (so existing apps are not broken), if it is not
specified the package is assumed to have version '0.0.0'
3. The existing 'Require' block of Application manifest will be used
to specify the package dependencies. Currently it is a yaml-based
dictionary, with the keys set to fully-qualified names of the dependency
packages and the values set to the version of those dependencies.
Currently
this block is used only for integration with apps stored at
apps.openstack.org. It is suggested to use this block in the
deployment process as well, and extend its semantics.
The version of the dependency specified there should also follow the
semver notation, however it may be specified in the shortened format, i.e.
without specifying the 'patch' or 'patch' and 'minior' components. In this
case the dependency will be specified as a range of allowed versions. For
example, a dependency version 1.2 will mean a (1.2.0 = version 1.3)
range.
If the version of a dependency is not specified (like in this
existing app - [1]) then we assume the version 0 - i.e. the last
available pre-release version of a package.
4. Murano core library is also a package which has its own version.
The current one is assumed to have a version 0.1.0, the one which is going
to be released in L will be probably called 0.2.0. The lib is still
quickly
evolving, so we are not releasing a 1.0.0 until we are sure that we are
not
going to have any breaking changes anytime soon.
As with any other package it will be possible to have several
versions of the Core Library installed in Murano at the same moment of
time.
5. There is no mandatory need to add the the dependency on the core
library to the Requires block of each application, as it is added there
implicitly. However, this implicit dependency will have a version 0 -
i.e. will reference the latest pre-release version of the Core Library
available. So it is still better to pin the core library requirement to a
particular
[openstack-dev] [Murano] Versioning of Murano packages and MuranoPL classes
query) as soon as that feature is implemented in Glance (currently only static dependencies are implemented there). Until that, the dependencies will be stored as a regular list of strings, and the Murano engine will process it and query Glance to fetch the packages. 9. In L cycle we are not going to show multiple versions of the same app in Murano dashboard: only the last one will be shown if the multiple versions are present. This is to minimize the changes at Dashboard side: in future releases we'll add the ability to select the proper version. The generation of the object model by dynamic UI also remains intact. 10. However, the structure of the object model isself gets changed: in the ? block of each object two new fields appear: package and version, which correspond to the FQN and the version of the package which contain the class of the given object. UI leaves these fields as Nones when it generates the OM, and the engine computes them in a regular way: queries the package repository for the most recent version of a package which contains the class with a given name, and saves information about its name and version. This values get persisted in an Object Model when it gets serialized after the deployment. As a result, the versions of the components are fixed once the environment is deployed, so if some packages get updated afterwards, the existing components remain pinned to their initial version. As a result, the environment may get several components of the same type but different versions. 11. When the Object Model is validated after the deserialization, the behavior of $.class() contract is changed. During its validation the value passed to the appropriate property or argument should be of a type which is declared either in a current package (or in the another version of the current package, given that the major component of the versions is the same) or in one of the packages satisfying the requirements of the current one. I.e. it becomes impossible to reference any class from the unreferenced package. 12. When inheriting some other class using the 'Extends' attribute, the ancestor class should be defined either in the current package or in one of the packages satisfying the requirements of the current one. 13. (creepy advanced stuff) It may turn out that in case of the multiple inheritance a single class will attempt to inherit from two different versions of a same class. An exception should be thrown in this case, unless there is a possibility to find a version of this class which satisfies all parties. *For example: classA inherits classB from packageX and classC from packageY. Both classB and classC inherit from classD from packageZ, however packageX depends on the version 1.2.0 of packageZ, while packageY depends on the version 1.3.0. This leads to a situation when classA transitively inherits classD of both versions 1.2 and 1.3. So, an exception will be thrown. However, if packageY's dependency would be just 1 (which means any of the 1.x.x family) the conflict would be resolved and a 1.2 would be used as it satisfies both inheritance chains.* So, all the above cover most of our present needs for MuranoPL package and class versioning. Also, we already have a way which allows us to properly version the format of MuranoPL language (a Format key in application manifests) and UI-definition files (Version key in that files). This basically allows us to target the packages for a minimum version of Murano / Murano Dashboard. I hope this rather lengthly email is useful. Stan Lagun has taken an action item to frame all the above into a more formal spec. [1] https://github.com/openstack/murano-apps/blob/master/MySQL/package/manifest.yaml#L26 -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Discuss simulated-execution-mode-murano-engine blueprint
Hi Gosha,
In my understanding, the base test class (io.murano.tests.TestFixture)
should have something like assertRaises method, similar to the one in python
--
Regards,
Alexander Tivelkov
On Mon, Jul 6, 2015 at 9:47 PM, Georgy Okrokvertskhov
gokrokvertsk...@mirantis.com wrote:
Hi Kate,
MuraPL has a concept of exceptions. Unit testing will be important to make
sure that workflows behaves correctly in exceptions situations. How will be
this addressed int he testing framework? Will you have some specific
assertions like python unit testing framework?
Thanks
Gosha
On Mon, Jul 6, 2015 at 9:02 AM, Ekaterina Chernova efedor...@mirantis.com
wrote:
Folks,
I have specific topic to discuss: how murano test-fixture will look like
and how it can be run.
The idea is to implement a unit-testing framework, similar to regular
frameworks for python or other languages,
which will allow to define test fixtures in MuranoPL.
A Test fixture is a regular muranoPL class definition, which methods are
the test cases.
When such fixture is included into Murano application package the
deployment of application may be tested without running actual VM's.
Here is how the test fixture may look like:
Namespaces:
=: io.murano.apps.foo.tests
sys: io.murano.system
pckg: io.murano.apps.foo
Extends: io.murano.tests.TestFixture
Name: FooTest
Methods:
initialize:
Body:
*# Object model can be loaded from json file, or provided
directly in murano-pl code as a yaml insertion.*
*# - $.appJson:
new(sys:Resources).json('foo-test-object-model.json')*
- $.appJson:
- ?:
type: io.murano.apps.foo.FooApp
name: my-foo-obj
instance:
?:
type: io.murano.resources.Instance
...
setUp:
Body:
- $.env: $.createEnvironment($.appJson) # creates an
instance of std:Environment
- $.myApp:
$.env.applications.where($.name='my-foo-obj').first()
*# mock instance spawning*
- mock($.myApp.instance, deploy, mockInstanceDeploy,
$this)
- mock(res:Instance, deploy, mockInstanceDeploy, $this)
testFooApp:
Body:
- $.env.deploy()
- $.assertEqual(true, $.myApp.getAttr('deployed'))
mockInstanceDeploy:
Arguments:
- mockContext
Body:
- Return:
* # heat template*
io.murano.tests.TestFixture - is a base class, that contains set of
methods, needed in all the test-cases which inherit it, such as assertEqual
and other similar assertions.
All it contains a $.createEnvironment method which may be called at the
setUp phase (a method being run before each test case) to construct the
object model to run the test against.
Test developer will be able to mock some of the functions or method which
are out of the scope of the current test
(for example, interaction with other applications or classes from the
standard murano library, such as instance.deploy etc).
The mock will allow to override the actual method execution and provide
the expected output of the method.
The actual implementation of mocking requires more thoughtful design,
so I'll create a separate spec for it.
What do you think about the overall idea and the test syntax proposed
above?
On Wed, Jun 3, 2015 at 5:15 PM, Ekaterina Chernova
efedor...@mirantis.com wrote:
Hi all!
I'd like to discuss first implementation thoughts about this [1]
blueprint, that we want to implement in Liberty.
This feature is supposed to increase the speed of application
development.
Now engine interacts with API to get input task and packages.
Items, planned to implement first would enable loading local task and
new package, without API and Rabbit involved.
After that simple testing machinery will be added to MuranoPL: mock
support and simple test-runner.
So user can test application methods as he wants by creating simple
tests.
Deployment parameters, such as heat stack and murano execution
plan outputs
may be set as returned value in tests.
Finally, tests may be placed into a murano package for easier package
verification and later modification.
I'm going to write specification soon. But before, we need to prepare
list of functions, that are needed to
implement simple mocking machinery in MuranoPL.
Please, leave your thoughts here or directly in the blueprint.
Regards, Kate.
[1] -
https://blueprints.launchpad.net/murano/+spec/simulated-execution-mode-murano-engine
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [murano] python versions
Hi Kirill, Client libraries usually have wider range of python requirements, as they may be run on various kinds of legacy environments, including the ones with python 2.6. only. Murano is definitely not the only project in Openstack which still maintains py26 compatibility for its client: nova, glance, cinder and other integrated ones do this. So, I would not drop py26 support for client code without any good reasons to. Are there any significant reasons to do it? Regarding py3.4 - this is definitely a good idea. -- Regards, Alexander Tivelkov On Tue, Jun 2, 2015 at 3:04 PM, Kirill Zaitsev kzait...@mirantis.com wrote: It seems that python-muranoclient is the last project from murano-official group, that still supports python2.6. Other projects do not have a 2.6 testing job (correct me if I’m wrong). Personally I think it’s time to drop support for 2.6 completely, and add (at least non-voting) jobs with python3.4 support tests. It seems to fit the whole process of moving OS projects towards python 3: https://etherpad.openstack.org/p/liberty-cross-project-python3 What do you think? Does anyone have any objections? -- Kirill Zaitsev Murano team Software Engineer Mirantis, Inc __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [new][app-catalog] App Catalog next steps
On Fri, May 29, 2015 at 7:55 PM, Fox, Kevin M kevin@pnnl.gov wrote: As an Op, I really really want to replace one image with a new one atomically with security updates preapplied. Think shellshock, ghost, etc. It will be basically be the same exact image as before, but patched. Referencing local ID's explicitly makes it harder to ensure things are patched, since new vm's will tend to pop up after things are patched with new vulnerabilities. That's the exact use case for the versioning concept we have in Artifacts: each artifact is identified by name and version, so there is always latest version of X and an API call which returns it. However, that's the question of different API calls and their proper usage: get-by-id returns the very same object which was uploaded, and get by name - the latest object matching the required version. First is good for bit-to-bit immutability guarantees, cache checks etc, second - for the use cases like yours. Same is true for the cross-artifact dependency relations: they may be static (i.e. reference by ID) or dynamic (i.e. reference by name and version). __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [new][app-catalog] App Catalog next steps
Hi Kevin, Has the Glance Artifact Repository implemented enough bits to have Heat and/or Murano artefacts yet? Most of the code is there already, couple of patchsets are still on review but we'll land them soon. L1 is a likely milestone to have it ready in master. Also, has there been any work on Exporting/Importing them through some defined format (tarball?) that doesn't depend on the artefact type? This one is not completely implemented: the design is ready (the spec had this feature from the very beginning) and a PoC was done. The final implementation is likely to happen in L cycle. I've been talking with the Heat folks on starting a blueprint to allow heat templates to use relative URL's instead of absolute ones. That would allow a set of Heat templates to be stored in one artefact in Glance. That's awesome. Also I'd consider allowing Heat to reference Templates by their artifact IDs in Glance, same as Nova does it for images. -- *From:* Alexander Tivelkov [ativel...@mirantis.com] *Sent:* Thursday, May 28, 2015 4:46 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [new][app-catalog] App Catalog next steps Hi folks, I believe that at least part of the filtering we are discussing here may be done at the client side if the client is sophisticated enough to be aware about the capabilities of the local cloud. And by sophisticated client I mean Glance V3 (previously known as Artifact Repository), which may (and, in my vision, should) become the ultimate consumer of the app catalog on the cloud side. Each asset type (currently Image, Murano Package, Heat template, more to come) should be implemented as Glance Artifact type (i.e. a plugin), and may define the required capabilities as its type specific metadata fields (for example, Heat-template type may list plugins which are required to run this template; Murano-package type may set the minimum required version of Core library etc). The logic which is needed to validate this capabilities may be put into this type-specific plugin as well. This custom logic method will gets executed when the artifact is being exported from app catalog into the particular cloud. In this case the compatibility of particular artifact with particular cloud will be validated by that cloud itself when the app catalog is browsed. Also, if the cloud does not have support of some artifact types at all (e.g. does not have Murano installed and thus cannot utilize Murano Packages), then it does not have the Murano plugin in its glance and thus will not be able to import murano-artifacts from the Catalog. Hope this makes sense. -- Regards, Alexander Tivelkov On Thu, May 28, 2015 at 10:29 AM, Morgan Fainberg morgan.fainb...@gmail.com wrote: On Wed, May 27, 2015 at 5:33 PM, Joe Gordon joe.gord...@gmail.com wrote: On Wed, May 27, 2015 at 4:27 PM, Fox, Kevin M kevin@pnnl.gov wrote: I'd say, tools that utilize OpenStack, like the knife openstack plugin, are not something that you would probably go to the catalog to find. And also, the recipes that you would use with knife would not be specific to OpenStack in any way, so you would just be duplicating the config management system's own catalog in the OpenStack catalog, which would be error prone. Duplicating all the chef recipes, and docker containers, puppet stuff, and . is a lot of work... I am very much against duplicating things, including chef recipes that use the openstack plugin for knife. But we can still easily point to external resources from apps.openstack.org. In fact we already do ( http://apps.openstack.org/#tab=heat-templatesasset=Lattice). The vision I have for the Catalog (I can be totally wrong here, lets please discuss) is a place where users (non computer scientists) can visit after logging into their Cloud, pick some app of interest, hit launch, and optionally fill out a form. They then have a running piece of software, provided by the greater OpenStack Community, that they can interact with, and their Cloud can bill them for. Think of it as the Apple App Store for OpenStack. Having a reliable set of deployment engines (Murano, Heat, whatever) involved is critical to the experience I think. Having too many of them though will mean it will be rare to have a cloud that has all of them, restricting the utility of the catalog. Too much choice here may actually be a detriment. calling this a catalog, which it sounds accurate, is confusing since keystone already has a catalog. Naming things is unfortunately a difficult problem. This is in itself turns into a really unfortunately usability issue for a number of reason; colliding namespaces that end users need to be aware of serves to generate confusion. Even the choices made naming things currently in use by OpenStack (I openly admit Keystone is particularly bad
Re: [openstack-dev] [new][app-catalog] App Catalog next steps
Hi Kevin, I don't suggest to use random IDs as artifact identifiers in the community app catalog. Of course we will need to have some globally unique names there (my initial idea on that is to have a combination of fully-qualified namespace-based name + version + signature) - and such names should be used to replicate artifacts across the cloud boundaries. By Referencing by ID I mean only the local referencing: when the artifact is already present in cloud's local Glance (be it imported from the community catalog, copied from other cloud or uploaded directly), the particular service (Heat in our example) should be able to consume it by ID, same as Nova currently does with Images. This has its own purpose to guarantee objects' immutability: once the user has selected an object in the local catalog, she may be sure that nobody will interfere and modify it, as the object itself is immutable and the id is not reusable. If the object is referenced only by name, then it may be deleted and a different artifact with the same name may be uploaded instead, which may introduce a potential security issue. Using IDs will prevent such behavior. Fully qualified object names are still needed, of course - but that's Glance goal to locate an artifact based on its FQN and return the id for it. At least, this was the design idea of the initial artifact concept. But that's an off-topic here, as this concept is related only to the local artifact repos, and world-global app catalog has nothing to do with this. -- Regards, Alexander Tivelkov On Fri, May 29, 2015 at 6:47 PM, Fox, Kevin M kevin@pnnl.gov wrote: Hi Alexander, Sweet. I'll have to kick the tires on the current state of Liberty soon. :) Reference by artifact IDs is going to be problematic I think. How do you release a generic set of resources to the world that reference specific randomly generated ID's? What about by Name? If not, then it will need to be some kind of mapping mechanism. :/ Thanks, Kevin -- *From:* Alexander Tivelkov [ativel...@mirantis.com] *Sent:* Friday, May 29, 2015 4:19 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [new][app-catalog] App Catalog next steps Hi Kevin, Has the Glance Artifact Repository implemented enough bits to have Heat and/or Murano artefacts yet? Most of the code is there already, couple of patchsets are still on review but we'll land them soon. L1 is a likely milestone to have it ready in master. Also, has there been any work on Exporting/Importing them through some defined format (tarball?) that doesn't depend on the artefact type? This one is not completely implemented: the design is ready (the spec had this feature from the very beginning) and a PoC was done. The final implementation is likely to happen in L cycle. I've been talking with the Heat folks on starting a blueprint to allow heat templates to use relative URL's instead of absolute ones. That would allow a set of Heat templates to be stored in one artefact in Glance. That's awesome. Also I'd consider allowing Heat to reference Templates by their artifact IDs in Glance, same as Nova does it for images. -- *From:* Alexander Tivelkov [ativel...@mirantis.com] *Sent:* Thursday, May 28, 2015 4:46 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [new][app-catalog] App Catalog next steps Hi folks, I believe that at least part of the filtering we are discussing here may be done at the client side if the client is sophisticated enough to be aware about the capabilities of the local cloud. And by sophisticated client I mean Glance V3 (previously known as Artifact Repository), which may (and, in my vision, should) become the ultimate consumer of the app catalog on the cloud side. Each asset type (currently Image, Murano Package, Heat template, more to come) should be implemented as Glance Artifact type (i.e. a plugin), and may define the required capabilities as its type specific metadata fields (for example, Heat-template type may list plugins which are required to run this template; Murano-package type may set the minimum required version of Core library etc). The logic which is needed to validate this capabilities may be put into this type-specific plugin as well. This custom logic method will gets executed when the artifact is being exported from app catalog into the particular cloud. In this case the compatibility of particular artifact with particular cloud will be validated by that cloud itself when the app catalog is browsed. Also, if the cloud does not have support of some artifact types at all (e.g. does not have Murano installed and thus cannot utilize Murano Packages), then it does not have the Murano plugin in its glance and thus will not be able to import murano-artifacts from
Re: [openstack-dev] [new][app-catalog] App Catalog next steps
Hi folks, I believe that at least part of the filtering we are discussing here may be done at the client side if the client is sophisticated enough to be aware about the capabilities of the local cloud. And by sophisticated client I mean Glance V3 (previously known as Artifact Repository), which may (and, in my vision, should) become the ultimate consumer of the app catalog on the cloud side. Each asset type (currently Image, Murano Package, Heat template, more to come) should be implemented as Glance Artifact type (i.e. a plugin), and may define the required capabilities as its type specific metadata fields (for example, Heat-template type may list plugins which are required to run this template; Murano-package type may set the minimum required version of Core library etc). The logic which is needed to validate this capabilities may be put into this type-specific plugin as well. This custom logic method will gets executed when the artifact is being exported from app catalog into the particular cloud. In this case the compatibility of particular artifact with particular cloud will be validated by that cloud itself when the app catalog is browsed. Also, if the cloud does not have support of some artifact types at all (e.g. does not have Murano installed and thus cannot utilize Murano Packages), then it does not have the Murano plugin in its glance and thus will not be able to import murano-artifacts from the Catalog. Hope this makes sense. -- Regards, Alexander Tivelkov On Thu, May 28, 2015 at 10:29 AM, Morgan Fainberg morgan.fainb...@gmail.com wrote: On Wed, May 27, 2015 at 5:33 PM, Joe Gordon joe.gord...@gmail.com wrote: On Wed, May 27, 2015 at 4:27 PM, Fox, Kevin M kevin@pnnl.gov wrote: I'd say, tools that utilize OpenStack, like the knife openstack plugin, are not something that you would probably go to the catalog to find. And also, the recipes that you would use with knife would not be specific to OpenStack in any way, so you would just be duplicating the config management system's own catalog in the OpenStack catalog, which would be error prone. Duplicating all the chef recipes, and docker containers, puppet stuff, and . is a lot of work... I am very much against duplicating things, including chef recipes that use the openstack plugin for knife. But we can still easily point to external resources from apps.openstack.org. In fact we already do ( http://apps.openstack.org/#tab=heat-templatesasset=Lattice). The vision I have for the Catalog (I can be totally wrong here, lets please discuss) is a place where users (non computer scientists) can visit after logging into their Cloud, pick some app of interest, hit launch, and optionally fill out a form. They then have a running piece of software, provided by the greater OpenStack Community, that they can interact with, and their Cloud can bill them for. Think of it as the Apple App Store for OpenStack. Having a reliable set of deployment engines (Murano, Heat, whatever) involved is critical to the experience I think. Having too many of them though will mean it will be rare to have a cloud that has all of them, restricting the utility of the catalog. Too much choice here may actually be a detriment. calling this a catalog, which it sounds accurate, is confusing since keystone already has a catalog. Naming things is unfortunately a difficult problem. This is in itself turns into a really unfortunately usability issue for a number of reason; colliding namespaces that end users need to be aware of serves to generate confusion. Even the choices made naming things currently in use by OpenStack (I openly admit Keystone is particularly bad in this light) have this issue. I would support a catalog-like name that limits confusion especially when it comes to conveying this information to the end users (not just deployers and operators). I will reiterate Joe's statement: Naming things is unfortunately a difficult problem. I respectfully disagree with this vision. I mostly agree with the first part about it being somewhere users can go to find applications that can be quickly deployed on OpenStack (note all the gotchas that Monty described here). The part I disagree with is about limiting the deployment engines to invented here. Even if we have 100 deployment engines on apps.openstack.org, it would be very easy for a user to filter by the deployment engines they use so I do not agree with your concern about too many choices here being a detriment (after all isn't OpenStack about choices?). ++ We should be as inclusive as we can be. There are many cases of prior art where (as long as it's workable) we can do filtering (someone brought up the mobile app stores). Even if we want to be measured in ensuring the filtering works before opening the flood gates, allowing alternate deployment engines is a good thing. It makes OpenStack more usable and more desirable as a platform
Re: [openstack-dev] [Glance] glance social event at Vancouver summit
Hi Brian, Seems like it overlaps with the Core reviewers party on Wednesday. Is there any chance to reschedule on Thursday? 15 мая 2015 г., в 7:19, Brian Rosmaita brian.rosma...@rackspace.com написал(а): Greetings to all Glance people, kragniz has proposed that we have an informal social event for Glance and Glance-related people attending the summit next week. Please keep your calendars open for about 2 hours or so starting at 7:30 p.m. on Wednesday, May 20. Neither kragniz nor I are familiar with Vancouver, so we'll figure out logistics when we get there and announce details during the Glance design sessions on Wednesday. Looking forward to seeing everyone and getting some good design work done next week ... safe travels! rosmaita __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance] Experimental API
Thanks Lakshmi, that's useful. So, we want to release Artifacts API in Kilo as experimental. We do need some early adopters to begin working with it (the initial interest was from Heat and Murano projects, and the OVA/OVF initiative for Images as well) in the next cycle and provide some feedback on the API and its usefulness, so we may take this feedback into account before releasing the stable version of API with L. I've talked to Murano folks, they are ok with that plan, some feedback from Heat and OVA teams would be great as well. Anyways, we will not break the compatibility without serious reasons for that, and we will collaborate with any early adopters about any such breaking changes. -- Regards, Alexander Tivelkov On Thu, Mar 12, 2015 at 8:19 PM, Sampath, Lakshmi lakshmi.samp...@hp.com wrote: We had a discussion with API WG today about what it means to be an EXPERIMENTAL API and here's the takeway from that discussion. - API's can be experimental, but mark it clearly in the docs as such - Experimental means a breaking change may be introduced - Use /x1/ instead of /v1/ in the endpoint. Thoughts/Suggestions? Thanks Lakshmi. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance] Proposal to change Glance meeting time.
Works for me, but the previous one worked as well. So, consider my vote as +1 unless the majority is against this :) -- Regards, Alexander Tivelkov On Mon, Mar 9, 2015 at 12:36 AM, Fei Long Wang feil...@catalyst.net.nz wrote: Oh, it means 3:00AM for me :-( On 09/03/15 09:07, Nikhil Komawar wrote: Hi all, Currently, we've alternating time for Glance meetings. Now, with the Daylight savings being implemented in some parts of the world, we're thinking of moving the meeting time to just one slot i.e. earlier in the day(night). This solves the original conflicting times issue that a subset of the individuals had; to add to that the schedule is less confusing and unified. So, the new proposal is: Glance meetings [1] to be conducted weekly on Thursdays at 1400UTC [2] on #openstack-meeting-4 This would be implemented on Mar 19th, given there are no major objections. Please vote with +1/-1 here. [1] https://wiki.openstack.org/wiki/Meetings#Glance_Team_meeting [2] http://www.timeanddate.com/worldclock/fixedtime.html?hour=14min=0sec=0 Thanks, -Nikhil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Cheers Best regards, Fei Long Wang (王飞龙) -- Senior Cloud Software Engineer Tel: +64-48032246 Email: flw...@catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington -- __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance] Glance K-3 reviews.
Thanks Nikhil, To have a single whiteboard for all the Artifacts reviews we've created an etherpad [1] Here we track the progress of all the 7 artifact-releated changesets and all the unresolved issues found there. As agreed earlier, non-critical issues (i.e. the issues which do not affect performance, security, API compatibility and do not cause HTTP 500 errors) should not block the review process: we log that issues in the etherpad to be fixed later as independent bugfix changesets or to be additionally discussed if such need exists. The critical ones are going to be fixed ASAP, of course. We are copying all the comments from reviews to that etherpad, but if you wish, please feel free to put notes there on your own if it is more convenient for you. Thanks for your reviews and feedback! [1] https://etherpad.openstack.org/p/Artifacts_Reviews -- Regards, Alexander Tivelkov On Sun, Mar 8, 2015 at 10:36 PM, Nikhil Komawar nikhil.koma...@rackspace.com wrote: Hi all, This is a gentle reminder on the ML regarding Kilo-3 related reviews. Similar announcements have been made in the past few weeks during the Glance meetings [2]. A small summary of what the current status looks like for k-3 Glance: 1. Feature freeze for Glance is Mar 12th. 2. [1] has the list of specs/BPs targeted for Kilo. (They are also our priorities for this cycle). 3. [1] should also give information regarding what needs to be reviewed. Many of the specs have a mature enough code; and a little chance for them to need major feedback. So, they are basically waiting on core reviewers to show up. 4. Please prefer to review specs over bugs at the moment as we can target bugs after Mar 12th before official freeze on Mar 19th (and some others during the RCs). 5. Only Artifacts, Catalog Index Service and Deactivate an Image specs have been proposed for possible FFE candidates (based on the momentum they have been having in discussions and commitment of members to get issues resolved). 6. We are planning to do a release of the client and store lib before the official freeze for accommodating some of the features. 7. If you need more details, please refer to [2] and look for logs of meetings on or after Feb 12th for K3 related updates. If you still have questions, then feel free to reach out. Please remember this is a busy period so, turn around time will be more. P.S. As some of the reviewers (and core-reviewers like Flavio and Zhi Yan) are not able to make it to the meetings often, this is a special announcement being made for their convenience. [1] https://launchpad.net/glance/+milestone/kilo-3 [2] http://eavesdrop.openstack.org/meetings/glance/2015/ Thanks, -Nikhil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Folks, one more thing to consider: the next big release of yaql (1.0,
coming really soon) will get support of curly braces (by default - to
initialise dictionaries in the following way:
v0.2/v0.3 syntax: dict(key1=value1, key2=value2)
v1 syntax: {key1=value1, key2=value2} (the old syntax works in v1 as well)
So, {} will become a valid yaql expression (empty dict). I believe it is
not a big deal to parse that correctly and differentiate between outer
markup and an expression contained within, however statements like {{}}
may be a little ugly.
--
Regards,
Alexander Tivelkov
On Thu, Feb 19, 2015 at 1:13 PM, Nikolay Makhotkin nmakhot...@mirantis.com
wrote:
Hi !
From those three I'd choose only { ... }, it looks better for YAML while
'%' sign looks foreign for YAML. Moreover, it needs extra spaces for
writing expressions:
Compare:
1. %$.var + 1%
2. % $.var + 1 %
3. {$.var + 1}
One more point from me: We can't do things just beacuse it is familiar
with N things and it should be so.
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance][Artifacts] Object Version format: SemVer vs pep440
Hi Rob, That is slightly different: from logical point of view that are different schemas indeed, however they all map to the same DB schema, so we do not have any issues with upgrades. There are some limitations on the schema modifications as well, and being able to work with multiple versions of the same artifact type is supported from the beginning and works quite well. -- Regards, Alexander Tivelkov On Mon, Feb 16, 2015 at 9:50 PM, Robert Collins robe...@robertcollins.net wrote: On 17 February 2015 at 03:31, Alexander Tivelkov ativel...@mirantis.com wrote: Hi Client, Thanks for your input. We actually support the scenarios you speak about, yet in a slightly different way. The authors of the Artifact Type (the plugin developers) may define their own custom field (or set of fields) to store their sequence information or any other type-specific version-related metadata. So, they may use generic version field (which is defined in the base artifact type) to store their numeric version - and use their type-specific field for local client-side processing. That sounds scarily like what Neutron did, leading to a different schema for every configuration. The reason Clint brought up Debian version numbers is that to sort them in a database you need a custom field type - e.g. http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/database/schema/launchpad-2209-00-0.sql#L25 . And thats quite a burden :) We've had fairly poor results with the Neutron variation in schemas, as it tightly couples things, making upgrades that change plugins super tricky, as well as making it very hard to concurrently support multiple plugins. I hope you don't mean you're doing the same thing :) -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance][Artifacts] Object Version format: SemVer vs pep440
Hi Ian, On Tue, Feb 10, 2015 at 11:17 PM, Ian Cordasco ian.corda...@rackspace.com wrote: I think the fundamental disconnect is that not every column in a database needs offer sorting to the user. Imposing that restriction here causes a cascade of further restrictions that will fundamentally cause this to be unusable by a large number of people. I didn't say that every column needs to offer sorting. However, ability to differentiate between different artifacts and different versions of the same artifact, as well as ability to sort and filter these different versions according to some criteria is one of the core features we were looking for when designing the whole artifact proposal. In fact, the initial request for this feature was not even made by me: I was asked about versions at the first design summit where we initially suggested artifacts (Icehouse mid-cycle) and in subsequent email and IRC discussions. The first design proposal which was presented in Atlanta already included this concept and so it was approved there, so this feature was always considered as a natural and important concept. I don't understand why it causes so much confusion now, when the implementation is almost complete. Except for the fact that versions do typically mean more than the values SemVer attaches to them. SemVer is further incompatible with any versioning scheme using epochs and is so relatively recent compared to versioning practices as a whole that I don’t see how we can justify restricting what should be a very generic system to something so specific to recent history and favored almost entirely by *developers*. I believe any person in software industry may invent their own versioning scheme - and most of them will be incompatible with each other. If you attempt to build something which is compatible with all of them at once, you will eventually end up having plain strings, without any semantic and precedence rules. This definitely does not satisfy our initial goal (see above). Instead, I suggest to choose the scheme which is strict, unambiguous and satisfies our initial goal, but has maximum possible adoption among the developers, so most of the adopters will be able to use it. Semver seems to be the best candidate here, but any other proposals are welcome as well. Meanwhile this does not prevent adopters from having their own schemas if they want it: Artifact Type developers may add their own type-specific string metadata field, put some regexp of code-based constraints on it - and use it to store their own version. Yet they will not be able to utilize the version-related features which are built-in into the Glance. Because up until now the only use case you’ve been referencing is CD software. There is some disconnect here: I believe the message which started this thread was the first time where I mentioned Artifact Repository in the context of Continuous Delivery solutions; and I was saying about CD system built on top of it, not about the Artifact Repostiory being a CD-system on its own. All other use-cases and scenarios did not mention CDs at all: Artifact Repostiory is definitely a generic catalog, not a CD solution. -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance][Artifacts] Object Version format: SemVer vs pep440
Donald, Thanks for your comments, really useful! I think I need to clarify a bit: I am not speaking about the actual semantic: placing the meaning into the actual values is still up to the end-users (or the developers of Artifact Types, if they build some custom logic which processes version info somehow). So, this thread is really about preferred syntax scheme - and the rules to determine precedence. I understand that pep440 has richer syntax with more capabilities (epochs, unlimited number of version segments, development releases etc). My only concern is that being a python-only standard it is less generic (in term of adoption) that the syntax of semver. The same goes to Monty's pbr semver: it is openstack-only and thus may be confusing. -- Regards, Alexander Tivelkov On Tue, Feb 10, 2015 at 11:32 PM, Donald Stufft don...@stufft.io wrote: On Feb 10, 2015, at 3:17 PM, Ian Cordasco ian.corda...@rackspace.com wrote: And of course, the chosen solution should be mappable to database, so we may do sorting and filtering on the DB-side. So, having it as a simple string and letting the user to decide what it means is not an option. Except for the fact that versions do typically mean more than the values SemVer attaches to them. SemVer is further incompatible with any versioning scheme using epochs and is so relatively recent compared to versioning practices as a whole that I don’t see how we can justify restricting what should be a very generic system to something so specific to recent history and favored almost entirely by *developers*. Semver vs PEP 440 is largely a syntax question since PEP 440 purposely does not have much of an opinion on how something like 2.0.0 and 2.1.0 are related other than for sorting. We do have operators in PEP 440 that support treating these versions in a semver like way, and some that support treating them in other ways. The primary purpose of PEP 440 was to define a standard way to parse and sort and specify versions across several hundred thouse versions that currently exist in PyPI. This means that it is more complicated to implement but it is much more powerful than semver eve could be. One example, as Ian mentioned is the lack of the ability to do an Epoch, another example is that PEP 440 has explicit support for someone taking version 1.0 adding some unofficial patches to it, and then releasing that in their own distribution channels. The primary purpose of Semver was to be extremely opinionated in what meaning you place on the *content* of the version parts and the syntax is really a secondary concern which exists just to make it easier to parse. This means that if you know ahead of time that something is Semver you can guess a lot more information about the relationship of two versions. It was our intention that PEP 440 would (is?) aimed primarily at people implementing tools that work with versions, and the additional PEPs or other documentations would be written on top of PEP 440 to add opinions on what a version looks like within the framework that PEP 440 sets up. A great example is the pbr semver document that Monty linked. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance][Artifacts] Object Version format: SemVer vs pep440
Hi Client, Thanks for your input. We actually support the scenarios you speak about, yet in a slightly different way. The authors of the Artifact Type (the plugin developers) may define their own custom field (or set of fields) to store their sequence information or any other type-specific version-related metadata. So, they may use generic version field (which is defined in the base artifact type) to store their numeric version - and use their type-specific field for local client-side processing. -- Regards, Alexander Tivelkov On Tue, Feb 10, 2015 at 11:37 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Alexander Tivelkov's message of 2015-02-10 07:28:55 -0800: Hi folks, One of the key features that we are adding to Glance with the introduction of Artifacts is the ability to have multiple versions of the same object in the repository: this gives us the possibility to query for the latest version of something, keep track on the changes history, and build various continuous delivery solutions on top of Artifact Repository. We need to determine the format and rules we will use to define, increment and compare versions of artifacts in the repository. There are two alternatives we have to choose from, and we are seeking advice on this choice. First, there is Semantic Versioning specification, available at [1]. It is a very generic spec, widely used and adopted in many areas of software development. It is quite straightforward: 3 mandatory numeric components for version number, plus optional string labels for pre-release versions and build metadata. And then there is PEP-440 spec, which is a recommended approach to identifying versions and specifying dependencies when distributing Python. It is a pythonic way to set versions of python packages, including PIP version strings. Conceptually PEP-440 and Semantic Versioning are similar in purpose, but slightly different in syntax. Notably, the count of version number components and rules of version precedence resolution differ between PEP-440 and SemVer. Unfortunately, the two version string formats are not compatible, so we have to choose one or the other. According to my initial vision, the Artifact Repository should be as generic as possible in terms of potential adoption. The artifacts were never supposed to be python packages only, and even the projects which will create and use these artifacts are not mandatory limited to be pythonic, the developers of that projects may not be python developers! So, I'd really wanted to avoid any python-specific notations, such as PEP-440 for artifacts. I've put this vision into a spec [3] which also contains a proposal on how to convert the semver-compatible version strings into the comparable values which may be mapped to database types, so a database table may be queried, ordered and filtered by the object version. So, we need some feedback on this topic. Would you prefer artifacts to be versioned with SemVer or with PEP-440 notation? Are you interested in having some generic utility which will map versions (in either format) to database columns? If so, which version format would you prefer? We are on a tight schedule here, as we want to begin landing artifact-related code soon. So, I would appreciate your feedback during this week: here in the ML or in the comments to [3] review. Hi. This is really interesting work and I'm glad Glance is growing into an artifact catalog as I think it will assist cloud users and UI development at the same time. It seems to me that there are really only two reasons to care about the content of the versions: sorting, and filtering. You want to make sure if people upload artifacts named myapp like this: myapp:1.0 myapp:2.0 myapp:1.1 That when they say show me the newest myapp they get 2.0, not 1.1. And if they say show me the newest myapp in the 1.x series they get 1.1. I am a little worried this is not something that can or should be made generic in a micro service. Here's a thought: You could just have the version, series, and sequence, and let users manage the sequencing themselves on the client side. This way if users want to use the _extremely_ difficult to program for Debian packaging version, you don't have to figure out how to make 1.0~special less than 1.0 and more than 0.9. To start with, you can have a default strategy of a single series, and max(sequence)+1000 if unspecified. Then teach the clients the various semvers/pep440's/etc. etc. and let them choose their own sequencing and series strategy. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List
Re: [openstack-dev] [glance] Cleanout of inactive change proposals from review
Hi! Important chagesets are supposed to have bugs (or blueprints) assigned to them, so, even if the CS is abandoned, its description still remains on Launchpad in one form or another, so we will not loose it from general project's backlog. And if the changeset didn't have a bug/blueprint specified, then it either does not represent a real use case at all (or its owner didn't bother to document it anyway, so keeping the changeset most probably will not help to understand the use case) So I like the proposal in general. However, it has a little issue: imagine a patchset which receives a -1 from some random reviewer. The owner may reply to that -1 with a reasonable counterargument, and in this situation it is up to the initial reviewer to either agree with that counterargument and revoke the -1 or to continue the discussion and persuade the owner to change the code. However, I've seen situations when the reviewers do not react to such replies and the changesets remain idle with a single -1 which are in fact addressed but not removed. It would be a bad practice if we abandon such commits just because their initial reviewers have lost any interest in continuing the discussion with owners. Probably we should keep such situations in mind when defining inactive. -- Regards, Alexander Tivelkov On Fri, Feb 13, 2015 at 5:17 PM, Kuvaja, Erno kuv...@hp.com wrote: Hi Boris, Thanks for your input. I do like the idea of picking up the changes that have not been active. Do you have resources in mind to dedicate for this? My personal take is that if some piece of work has not been touched for a month, it’s probably not that important after all and the community should use the resources to do some work that has actual momentum. The changes itself will not disappear the owner is still able to revive it if felt that there is right time to continue it. The cleanup will just make it easier for people to focus on things that are actually moving. It also will make bug tracking bit easier when one will see on the bug report that the patch got abandoned due to inactivity and indicates that the owner of that bug might not be working on it after all. - Erno From: Boris Pavlovic [mailto:bpavlo...@mirantis.com] Sent: Friday, February 13, 2015 1:25 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [glance] Cleanout of inactive change proposals from review Hi, I believe that keeping review queue clean is the great idea. But I am not sure that set of these rules is enough to abandon patches. Recently I wrote blogpost related to making OpenStack community more user friendly: http://boris-42.me/thoughts-on-making-openstack-community-more-user-friendly/ tl;dr; Patches on review are great source of information what is missing in project. Removing them from queue means losing this essential information. The result of such actions is that project doesn't face users requirements which is quite bad... What if that project team continue work on all abandoned patches that are covering valid use cases and finish them? Best regards, Boris Pavlovic On Fri, Feb 13, 2015 at 3:52 PM, Flavio Percoco fla...@redhat.com wrote: On 13/02/15 11:06 +, Kuvaja, Erno wrote: Hi all, We have almost year old (from last update) reviews still in the queue for glance. The discussion was initiated on yesterday’s meeting for adopting abandon policy for stale changes. The documentation can be found from https://etherpad.openstack.org/p/ glance-cleanout-of-inactive-PS and any input would be appreciated. For your convenience current state below: Thanks for putting this together. I missed the meeting yday and this is important. Glance - Cleanout of inactive change proposals from review We Should start cleaning out our review list to keep the focus on changes that has momentum. Nova is currently abandoning change proposals that has been inactive for 4 weeks. Proposed action (if all of the following is True, abandon the PS): 1. The PS has -1/-2 (including Jenkins) I assume you're talking about voting -1/-2 and not Workflow, right? (you said jenkins afterall but just for the sake of clarity). 2. The change is proposed to glance, glance_store or python-glanceclient; specs should not be abandoned as their workflow is much slower 3. No activity for 28 days from Author/Owner after the -1/-2 I'd reword this in No activity. This includes comments, feedback, discussions and or other committers taking over a patch. 4. There has been query made to the owner to update the patch between 5 and 10 days before abandoning (comment on PS/Bug or something similar) ● Let's be smart on this. Flexibility is good on holiday seasons, during feature freeze, etc. +2 to the above, I like it. Thanks again, Flavio -- @flaper87 Flavio Percoco
[openstack-dev] [Glance][Artifacts] Object Version format: SemVer vs pep440
Hi folks, One of the key features that we are adding to Glance with the introduction of Artifacts is the ability to have multiple versions of the same object in the repository: this gives us the possibility to query for the latest version of something, keep track on the changes history, and build various continuous delivery solutions on top of Artifact Repository. We need to determine the format and rules we will use to define, increment and compare versions of artifacts in the repository. There are two alternatives we have to choose from, and we are seeking advice on this choice. First, there is Semantic Versioning specification, available at [1]. It is a very generic spec, widely used and adopted in many areas of software development. It is quite straightforward: 3 mandatory numeric components for version number, plus optional string labels for pre-release versions and build metadata. And then there is PEP-440 spec, which is a recommended approach to identifying versions and specifying dependencies when distributing Python. It is a pythonic way to set versions of python packages, including PIP version strings. Conceptually PEP-440 and Semantic Versioning are similar in purpose, but slightly different in syntax. Notably, the count of version number components and rules of version precedence resolution differ between PEP-440 and SemVer. Unfortunately, the two version string formats are not compatible, so we have to choose one or the other. According to my initial vision, the Artifact Repository should be as generic as possible in terms of potential adoption. The artifacts were never supposed to be python packages only, and even the projects which will create and use these artifacts are not mandatory limited to be pythonic, the developers of that projects may not be python developers! So, I'd really wanted to avoid any python-specific notations, such as PEP-440 for artifacts. I've put this vision into a spec [3] which also contains a proposal on how to convert the semver-compatible version strings into the comparable values which may be mapped to database types, so a database table may be queried, ordered and filtered by the object version. So, we need some feedback on this topic. Would you prefer artifacts to be versioned with SemVer or with PEP-440 notation? Are you interested in having some generic utility which will map versions (in either format) to database columns? If so, which version format would you prefer? We are on a tight schedule here, as we want to begin landing artifact-related code soon. So, I would appreciate your feedback during this week: here in the ML or in the comments to [3] review. Thanks! [1] www.semver.org [2] www.python.org/dev/peps/pep-0440/ [3] https://review.openstack.org/#/c/139595/ -- Regards, Alexander Tivelkov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance][Artifacts] Object Version format: SemVer vs pep440
Hi Ian, Automatic version generation is not the only and not the primary reason for the version concept. In fact, the implementation which is planned to land in this cycle does not contain this feature at all: currently we also leave the version assignment up to uploader (version is a regular immutable generic artifact property). Auto-increments as part of clone-and-modify scenarios are postponed for the next cycle. However, even now we do need to have some sorting order - so, we need rules to determine precedence. That's the reason for having some notation defined: if we leave the notation up to the end-user we won't be able to compare artifacts having versions in different notations. And we even can't leave it up to the Artifact Type developer, since this is a generic property, thus common for all the artifact types. And of course, the chosen solution should be mappable to database, so we may do sorting and filtering on the DB-side. So, having it as a simple string and letting the user to decide what it means is not an option. Speaking about Artifactory - that's entirely different thing. It is indeed a continuous delivery solution, composed around build machines, deployment solutions and CI systems. That's definitely not what Glance Artifact Repository is. Even the concepts of Artifact are entirely different. So, while Artifact Repository may be used to build some CD solutions on top of it (or to be integrated with the existing ones) it is not a storage solution for build outputs and thus I can barely see how we may compare them. -- Regards, Alexander Tivelkov On Tue, Feb 10, 2015 at 8:15 PM, Ian Cordasco ian.corda...@rackspace.com wrote: On 2/10/15, 10:35, Alexander Tivelkov ativel...@mirantis.com wrote: Thanks Monty! Yup, probably I've missed that. I was looking at pbr and its version implementation, but didn't realize that this is actually a fusion of semver and pep440. So, we have this as an extra alternative to choose from. It would be an obvious choice if we were just looking for some common solution to version objects within openstack. However, I am a bit concerned about applying it to Artifact Repository. As I wrote before, we are trying to make the Repository to be language- and platform-agnostic tool for other developers, including the ones originating from non-python and non-openstack worlds. Having a versioning notation which is non-standard for everybody but openstack developers does not look like a good idea to me. -- Regards, Alexander Tivelkov On Tue, Feb 10, 2015 at 6:55 PM, Monty Taylor mord...@inaugust.com wrote: On 02/10/2015 10:28 AM, Alexander Tivelkov wrote: Hi folks, One of the key features that we are adding to Glance with the introduction of Artifacts is the ability to have multiple versions of the same object in the repository: this gives us the possibility to query for the latest version of something, keep track on the changes history, and build various continuous delivery solutions on top of Artifact Repository. We need to determine the format and rules we will use to define, increment and compare versions of artifacts in the repository. There are two alternatives we have to choose from, and we are seeking advice on this choice. First, there is Semantic Versioning specification, available at [1]. It is a very generic spec, widely used and adopted in many areas of software development. It is quite straightforward: 3 mandatory numeric components for version number, plus optional string labels for pre-release versions and build metadata. And then there is PEP-440 spec, which is a recommended approach to identifying versions and specifying dependencies when distributing Python. It is a pythonic way to set versions of python packages, including PIP version strings. Conceptually PEP-440 and Semantic Versioning are similar in purpose, but slightly different in syntax. Notably, the count of version number components and rules of version precedence resolution differ between PEP-440 and SemVer. Unfortunately, the two version string formats are not compatible, so we have to choose one or the other. According to my initial vision, the Artifact Repository should be as generic as possible in terms of potential adoption. The artifacts were never supposed to be python packages only, and even the projects which will create and use these artifacts are not mandatory limited to be pythonic, the developers of that projects may not be python developers! So, I'd really wanted to avoid any python-specific notations, such as PEP-440 for artifacts. I've put this vision into a spec [3] which also contains a proposal on how to convert the semver-compatible version strings into the comparable values which may be mapped to database types, so a database table may be queried, ordered and filtered by the object version. So, we need some feedback on this topic. Would you prefer artifacts to be versioned with SemVer or with PEP-440 notation
Re: [openstack-dev] [Glance][Artifacts] Object Version format: SemVer vs pep440
Thanks Monty! Yup, probably I've missed that. I was looking at pbr and its version implementation, but didn't realize that this is actually a fusion of semver and pep440. So, we have this as an extra alternative to choose from. It would be an obvious choice if we were just looking for some common solution to version objects within openstack. However, I am a bit concerned about applying it to Artifact Repository. As I wrote before, we are trying to make the Repository to be language- and platform-agnostic tool for other developers, including the ones originating from non-python and non-openstack worlds. Having a versioning notation which is non-standard for everybody but openstack developers does not look like a good idea to me. -- Regards, Alexander Tivelkov On Tue, Feb 10, 2015 at 6:55 PM, Monty Taylor mord...@inaugust.com wrote: On 02/10/2015 10:28 AM, Alexander Tivelkov wrote: Hi folks, One of the key features that we are adding to Glance with the introduction of Artifacts is the ability to have multiple versions of the same object in the repository: this gives us the possibility to query for the latest version of something, keep track on the changes history, and build various continuous delivery solutions on top of Artifact Repository. We need to determine the format and rules we will use to define, increment and compare versions of artifacts in the repository. There are two alternatives we have to choose from, and we are seeking advice on this choice. First, there is Semantic Versioning specification, available at [1]. It is a very generic spec, widely used and adopted in many areas of software development. It is quite straightforward: 3 mandatory numeric components for version number, plus optional string labels for pre-release versions and build metadata. And then there is PEP-440 spec, which is a recommended approach to identifying versions and specifying dependencies when distributing Python. It is a pythonic way to set versions of python packages, including PIP version strings. Conceptually PEP-440 and Semantic Versioning are similar in purpose, but slightly different in syntax. Notably, the count of version number components and rules of version precedence resolution differ between PEP-440 and SemVer. Unfortunately, the two version string formats are not compatible, so we have to choose one or the other. According to my initial vision, the Artifact Repository should be as generic as possible in terms of potential adoption. The artifacts were never supposed to be python packages only, and even the projects which will create and use these artifacts are not mandatory limited to be pythonic, the developers of that projects may not be python developers! So, I'd really wanted to avoid any python-specific notations, such as PEP-440 for artifacts. I've put this vision into a spec [3] which also contains a proposal on how to convert the semver-compatible version strings into the comparable values which may be mapped to database types, so a database table may be queried, ordered and filtered by the object version. So, we need some feedback on this topic. Would you prefer artifacts to be versioned with SemVer or with PEP-440 notation? Are you interested in having some generic utility which will map versions (in either format) to database columns? If so, which version format would you prefer? We are on a tight schedule here, as we want to begin landing artifact-related code soon. So, I would appreciate your feedback during this week: here in the ML or in the comments to [3] review. Because you should have more things to look at: http://docs.openstack.org/developer/pbr/semver.html We've already done some work to try to reconcile the world of semver with the world of PEP440 over in PBR land. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Nominating Kate Chernova for murano-core
+2! Need moar good cores! -- Regards, Alexander Tivelkov On Thu, Dec 25, 2014 at 12:50 PM, Stan Lagun sla...@mirantis.com wrote: +2 Sincerely yours, Stan Lagun Principal Software Engineer @ Mirantis On Thu, Dec 25, 2014 at 12:42 PM, Ruslan Kamaldinov rkamaldi...@mirantis.com wrote: Great addition to core team! +2 On Thu, Dec 25, 2014 at 11:02 AM, Timur Sufiev tsuf...@mirantis.com wrote: +1 from me. On Thu, Dec 25, 2014 at 10:28 AM, Serg Melikyan smelik...@mirantis.com wrote: I'd like to propose that we add Kate Chernova to the murano-core. Kate is active member of our community for more than a year, she is regular participant in our IRC meeting and maintains a good score as contributor: http://stackalytics.com/report/users/efedorova Please vote by replying to this thread. As a reminder of your options, +1 votes from 5 cores is sufficient; a -1 is a veto. -- Serg Melikyan, Senior Software Engineer at Mirantis, Inc. http://mirantis.com | smelik...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Timur Sufiev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Glance][Artifacts] Glance Artifacts Repository as a standalone service
Hi stackers, It has been some time since the announcement of Artifacts initiative within the Glance. The feature was not complete in Juno, but is being actively developed now and has good chances for landing in Kilo. However, recently on the Glance Virtual Mini-summit we had a discussions which lead to an idea to change one of the core design concepts of the Glance Artifact repository [1] Initially we planned to run Artifact repository as part of existing Glance service(s): all the APIs to handle artifacts were supposed to be hosted by glance-api service, with glance-registry as optional backend. Artifact-related endpoints were designed to be in the /v2/ branch of the API hierarchy, and were supposed to be similar in syntax and semantics to the existing v2/images endpoints. Now it is suggested to host artifacts as a standalone process listening to a different port (and probably deployed on a different host) and registered in the keystone as a separate service type. The code will be still part of the primary Glance repo - so this is not the question of starting another project or program, this is just about having a new daemon in the openstack deployment. This approach has some obvious pros and some less-obvious cons. Most important is the ability to load-balance images and artifacts independenly, being sure that any load on artifacts repo does no affect the performance of images - and vice versa. Also, this will allow us to provide different configuration options (including different backing storages) to these different components which will increase the overall flexibility and customizability of the system. We'll be able to design the artifacts API from scratch without the need to comply with the existing semantics and architecture of images-part, reusing only the components which are really needed. At the same time we'll loose the transparency between the concepts of image and artifact: initially we planned to make them very similar, so when we are finally ready to make images just one of the available artifact types, the migration may be trivial. If we now separate them into different services, we draw a strict border and potentially complicate the migration. IMHO, the pros outweigh the cons, so I personally like the idea of service separation - and all the participants of our virtual summit seemed to share the same opinion. However, it is a serious design change, so I'd like to hear the opinions of the wider audience. We have planned a design session in Paris ([2]) to discuss this change in more details (the topic is applicable not only to Artifacta, but to other initiatives under the hood of Glance as well, e.g. metadef catalog, index service etc) - please feel free to join and discuss. And please do not hesitate to share any concerns in the mailing list before the summit starts: the more opinions we have, the better solution we will make. [1] https://etherpad.openstack.org/p/kilo-glance-artifacts-current-state-of-development [2] https://etherpad.openstack.org/p/kilo-glance-summit-topics-final -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Glance] Concurrent update issue in Glance v2 API
Hi folks! There is a serious issue [0] in the v2 API of Glance which may lead to race conditions during the concurrent updates of Images' metadata. It can be fixed in a number of ways, but we need to have some solution soon, as we are approaching rc1 release, and the race in image updates looks like a serious problem which has to be fixed in J, imho. A quick description of the problem: When the image-update is called (PUT /v2/images/%image_id%/) we get the image from the repository, which fetches a record from the DB and forms its content into an Image Domain Object ([1]), which is then modified (has its attributes updated) and passed through all the layers of our domain model. This object is not managed by the SQLAlchemy's session, so the modifications of its attributes are not tracked anywhere. When all the processing is done and the updated object is passed back to the DB repository, it serializes all the attributes of the image into a dict ([2]) and then this dict is used to create an UPDATE query for the database. As this serialization includes all the attribute of the object (rather then only the modified ones), the update query updates all the columns of the appropriate database row, putting there the values which were originally fetched when the processing began. This may obviously overwrite the values which could be written there by some other concurent request. There are two possible solutions to fix this problem. First, known as the optimistic concurrency control, checks if the appropriate database row was modified between the data fetching and data updates. In case of such modification the update operation reports a conflict and fails (and may be retried based on the updated data if needed). Modification detection is usually based on the timstamps, i.e. the query updates the row in database only if the timestamp there matches the timestamp of initially fetched data. I've introduced this approach in this patch [3], however it has a major flaw: I used the 'updated_at' attribute as a timestamp, and this attribute is mapped to a DateTime-typed column. In many RDBMS's (including MySql5.6.4) this column stores values with per-second precision and does not store fractions of seconds. So, even if patch [3] is merged the race conditions may still occur if there are many updates happening at the same moment of time. A better approach would be to add a new column with int (or longint) type to store millisecond-based (or even microsecond-based) timestamps instead of (or additionally to) date-time based updated_at. But data model modification will require to add new migration etc, which is a major step and I don't know if we want to make it so close to the release. The second solution is to keep track of the changed attributes and properties for the image and do not include the unchanged ones into the UPDATE query, so nothing gets overwritten. This dramatically reduces the threat of races, as the updates of different properties do not interfere with each other. Also this is a usefull change regardless of the race itself: being able to differentiate between changed and unchanged attributes may have its own value for other purposes; the DB performance will also be better when updating just the needed fields instead of all of them. I've submitted a patch with this approach as well [4], but it still breaks some unittests and I am working to fix them right now. So, we need to decide which of these approaches (or their combination) to take: we may stick with optimistic locking on timestamp (and then decide if we are ok with a per-second timestamps or we need to add a new column), choose to track state of attributes or combine them together. So, could you folks please review patches [3] and [4] and come up with some ideas on them? Also, probably we should consider targeting [0] to juno-rc1 milestone to make sure that this bug is fixed in J. Do you guys think it is possible at this stage? Thanks! [0] https://bugs.launchpad.net/glance/+bug/1371728 [1] https://github.com/openstack/glance/blob/master/glance/db/__init__.py#L74 [2] https://github.com/openstack/glance/blob/master/glance/db/__init__.py#L169 [3] https://review.openstack.org/#/c/122814/ [4] https://review.openstack.org/#/c/123722/ -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano]
Hi Steve, Sorry I've missed this discussion for a while, but it looks like I have to add my 5 cents here now. Initially our intension was to make each Murano component self deployable, i.e. to incapsulate within its deploy method all the necessary actions to create the component, including generation of Heat snippet, merging it to the environment's template, pushing this template to Heat and doing any post-heat configuration if needed via Murano Agent. That's why the deploy method of NeutronNetwork class is doing $.environment.stack.push() - to make sure that the network is created when this method is called, regardless of the usages of this network in other components of the Environment. If you remove it from there, the call to network.deploy() will simply update the template in the environment.stack, but the actual update will not happen. So, the deploy method will not actually deploy anything - it will just prepare some snippet for future pushing. I understand your concerns though. But probably the solution should be more complex - and I like the idea of having event-based workflow proposed by Stan above. I even don't think that we do really need the deploy() methods in the Apps or Components. Instead, I suggest to have more fine-grained workflow steps which are executed by higher-level entity , such as Environment. For example, heat-based components may have createHeatSnippet() methods which just return a part of the heat template corresponding to the component. The deploy method of the environment may iteratively process all its components (and their nested components as well, of course), call this createHeatSnippet methods, merge the results into a single template - and then push this template as a single call to Heat. Then a post-heat config phase may be executed, if needed to run something with Murano Agent (as Heat Software Config is now the recommended way to deploy the software, there should be not too many of such needs - only for Windows-based deployments and other legacy stuff). -- Regards, Alexander Tivelkov On Tue, Jul 22, 2014 at 2:59 PM, Lee Calcote (lecalcot) lecal...@cisco.com wrote: Gents, For what it’s worth - We’ve long accounting for “extension points” within our VM and physical server provisioning flows, where developers may drop in code to augment OOTB behavior with customer/solution-specific needs. While there are many extension points laced throughout different points in the provisioning flow, we pervasively injected “pre” and “post” provisioning extension points to allow for easy customization (like the one being attempted by Steve). The notions of prepareDeploy and finishDeploy resonant well. Regards, Lee *Lee Calcote* * Sr. Software Engineering Manager Cloud and Virtualization Group * Phone: 512-378-8835 Mail/Jabber/Video: *lecal...@cisco.com* United States www.cisco.com From: Stan Lagun sla...@mirantis.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: Tuesday, July 22, 2014 at 4:37 PM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Murano] Hi Steve, 1. There are no objections whatsoever if you know how to do it without breaking the entire concept 2. I thing that deployment workflow need to be broken to more fine-grained steps. Maybe instead of single deploy methdos have prepareDeploy (which doesn't push the changes to Heat), deploy and finishDeploy. Maybe more/other methods need to be defined. This will make the whole process more customizible 3. If you want to have single-instance applications based on a fixed prebuild image then maybe what you need is to have your apps inhertir both Application and Instance classes and then override Instance's deploy method and add HOT snippet before VM instantiation. This may also require ability for child class to bind fixed values to parent class properties (narrowing class public contract, hiding those properties from user). This is not yet supported in MuranoPL but can be done in UI form as a temporary workaround 4. Didn't get why you mentioned object model. Object model is mostly user input. Do you suggest passing HOT snippets as part of user input? If so that would be something I oppose to 5. I guess image tagging would be better solution to image-based deployment 6. Personally I believe that problem can be eficently solved by Murano today or in the nearest future without resorting to pure HOT packages. This is not against Murano design and perfectly alligned with it Sincerely yours, Stan Lagun Principal Software Engineer @ Mirantis sla...@mirantis.com On Tue, Jul 22, 2014 at 8:05 PM, McLellan, Steven steve.mclel...@hp.com wrote: Hi, This is a little rambling, so I’ll put this summary here and some discussion below. I would like to be able to add heat template
Re: [openstack-dev] [Glance] Anyone using owner_is_tenant = False with image members?
Thanks Scott, that is a nice topic In theory, I would prefer to have both owner_tenant and owner_user to be persisted with an image, and to have a policy rule which allows to specify if the users of a tenant have access to images owned by or shared with other users of their tenant. But this will require too much changes to the current object model, and I am not sure if we need to introduce such changes now. However, this is the approach I would like to use in Artifacts. At least the current version of the spec assumes that both these fields to be maintained ([0]) [0] https://review.openstack.org/#/c/100968/4/specs/juno/artifact-repository.rst -- Regards, Alexander Tivelkov On Thu, Jul 3, 2014 at 3:44 AM, Scott Devoid dev...@anl.gov wrote: Hi folks, Background: Among all services, I think glance is unique in only having a single 'owner' field for each image. Most other services include a 'user_id' and a 'tenant_id' for things that are scoped this way. Glance provides a way to change this behavior by setting owner_is_tenant to false, which implies that owner is user_id. This works great: new images are owned by the user that created them. Why do we want this? We would like to make sure that the only person who can delete an image (besides admins) is the person who uploaded said image. This achieves that goal nicely. Images are private to the user, who may share them with other users using the image-member API. However, one problem is that we'd like to allow users to share with entire projects / tenants. Additionally, we have a number of images (~400) migrated over from a different OpenStack deployment, that are owned by the tenant and we would like to make sure that users in that tenant can see those images. Solution? I've implemented a small patch to the is_image_visible API call [1] which checks the image.owner and image.members against context.owner and context.tenant. This appears to work well, at least in my testing. I am wondering if this is something folks would like to see integrated? Also for glance developers, if there is a cleaner way to go about solving this problem? [2] ~ Scott [1] https://github.com/openstack/glance/blob/master/glance/db/sqlalchemy/api.py#L209 [2] https://review.openstack.org/104377 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Nominations to Murano core
+1 on both Serge and Steve -- Regards, Alexander Tivelkov On Thu, Jun 26, 2014 at 1:37 PM, Ruslan Kamaldinov rkamaldi...@mirantis.com wrote: I would like to nominate Serg Melikyan and Steve McLellan to Murano core. Serge has been a significant reviewer in the Icehouse and Juno release cycles. Steve has been providing consistent quality reviews and they continue to get more frequent and better over time. Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Constraint validation and property list filtering in Murano
Hi folks, There is an important topic which I would like to discuss: it seems like there is a place for improvement in UI validation and filtering in Murano. The reason of writing this is a change-set [1] (being an implementation of blueprint [2]) which allows package developers to specify the constraints for Flavor fields in dynamic UI definitions, and a little controversy about this commit among the core team. In my opinion, the change itself is great (thanks, Ryan!) and I am going to put my +2 on it, but I would like to say that there may exist a better and more complete approach, which we probably should adopt in future. The main idea is that in Murano we have a concept of Application Definitions, and these definitions should be complete enough to specify all the properties, dependencies, constraints and limitations for each application in the Catalog. Currently we write these defintions in MuranoPL, and the constraints and limitations are defined as its Contracts. For example, imagine we have an application which should be run on a Server having some specific hardware spec, e.g. having not less then 2 CPU cores and at least 8 Gb of RAM. In this case, these limits may be expressed as the Contract on the property defining the reference to the VM. The contract may look like this: $.class(Instance).check($.flavor.cpuCores=2 and $.flavor.ramMb=8192) (this will require us to create a data structure for flavors: currently we use plain string names - but this is quite an easy and straitforward change) Defining filter constraints on the UI side without having them in MuranoPL constraints is not enough: even if the UI is used to restrict the values of some properties, these restrictions may be ignored if the input object mode is composed manually and is sent to MuranoAPI without UI usage. This means that the MuranoPL contract should be the primary source of constraints/limitations, while the UI-side properties only suppliment them. This causes the need of defining constraints in two locations: in MuranoPL for runtime validation and in UI definitions for client-side checks and filtering. These two have different notations: MuranoPL uses flexible yaql-based contracts, which allow to construct and enforce almost eny expressions, while DynamicUI has a limited number of available properties for each type of input field. If some field does not have ability to enforce some check, then it has to be added in python code and commited to Murano's codebase, which contradicts with the mission of Application Catalog. This approach is overcomplicated, as it requires the package developer to learn two different notations. Also it is error-prone, as there is no automatic way to ensure that the ui-side constraint definitions do really match the MuranoPL contracts. So, I would prefer to have a single location of constraint definitions - MuranoPL contracts. These contracts (in their yaql form) should be processible by the dynamic UI and should be used for both field value checks and dropdown lists filterings. Also, the UI form for each component of the environment should be displayed and validated in the context of the contract applied to this component. In the example given above, the virtual machine contract is defined for the application class, while the UI form for it is defined for Instance class. While this form should be the same in all usages of this class, its context (availability and possible values of different fields) should be defined by the contracts defined by the class which uses it, i.e. the Application. As a bottom line, I would suggest to accept commit [1] for now (we need flavor filtering anyway), but agree that this should be a temporary workaround. Meahwile, we need to design and implement a way of passing contracts from MuranoPL classes to the UI engine and use this contracts fro both API-side validation and list filtering. [1] https://review.openstack.org/#/c/97904/ [2] https://blueprints.launchpad.net/murano/+spec/filter-flavor-for-each-service -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Murano API improvements
+2 to Stan: there is a strong need for v2 API which would satisfy the needs of Murano 0.5+. The current one is definitely outdated. I'd like to point that there are at least two major flaws in the v1 design: 1. Sessions. Actually, they violate one of the core principles of RESTful design - the client's state should not be stored on server, while our session implicitly do so. I would suggest to drop client-bound sessions entirely and instead introduce a concept of environment modification drafts: each environment may have an associated draft containing changes which are currently pending. These changes may include new components which have to be added to environment or modified copies of already existing components which have to be changed. There should be just one draft per environment, and any user of the tenant may see it, interact with it, apply it (i.e. run the deployment) etc. When the deployment is started, the draft is blocked (i.e. nobody can plan any other changes), and when it finishes the new configuration is saved within the environment, and the draft is cleaned up. In UI this may be implemented as having two tables for components view of the environment, where one of the table contains current state of the environment, while the second lists the Pending changes. This is just s brief suggestions on this topic, if there are other opinions, please post them here. Or should we create an etherpad to discuss it more actively? 2. Currently our API acts like an interactive JSON editor: it allows to crated any arbitrary nested JSON structures. Instead, it should act as an interactive editor of the valid Murano object model, and the API itself should be aware of Murano's specific: i.e. it should be able to differentiate between nesting objects and setting a link to the object existing at other model location, validate contracts etc. Also there was an idea of introducing a MuranoPL wizards to init the object model. This worths a separate email, but, briefly speaking, there should be a way to define some MuranoPL code which will construct a complex Murano object model based on the simple input, and this code has to be executed at API side. This will also require significant modifications of the API and making it aware of available MuranoPL wizard initializers and their semantics. So, this will require a significant modification of API, which means we have to design and deliver a v2 API spec. -- Regards, Alexander Tivelkov On Mon, Jun 2, 2014 at 1:06 PM, Stan Lagun sla...@mirantis.com wrote: I think API need to be redesigned at some point. There is a blueprint for this: https://blueprints.launchpad.net/murano/+spec/api-vnext It seems reasonable to implement new API on new framework at once Sincerely yours, Stan Lagun Principal Software Engineer @ Mirantis sla...@mirantis.com On Mon, Jun 2, 2014 at 12:21 PM, Ruslan Kamaldinov rkamaldi...@mirantis.com wrote: Let's follow the standard procedure. Both blueprints lack specification of implementation details. There also has to be someone willing to implement these blueprints in near feature. I'm not opposed to these ideas and I'd really like to see Pecan added during Juno, but we still need to follow the procedure. I cannot approve an idea, it should be a specification. Let's work together on the new API specification first, then we'll need to find a volunteer to implement it on top of Pecan. -- Ruslan On Mon, Jun 2, 2014 at 8:35 AM, Timur Nurlygayanov tnurlygaya...@mirantis.com wrote: Hi all, We need to rewrite Murano API on new API framework and we have the commit: https://review.openstack.org/#/c/60787 (Sergey, sorry, but -1 from me, need to fix small isses) Also, today I created blueprint: https://blueprints.launchpad.net/murano/+spec/murano-api-workers this feature allows to run many API threads on one host and this allows to scale Murano API processes. I suggest to update and merge this commit with migration to Pecan framework and after that we can easily implement this blueprint and add many other improvements to Murano API and Murano python agent. Ruslan, could you please approve these blueprints and target them to some milestone? Thank you! -- Timur, QA Engineer OpenStack Projects Mirantis Inc ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http
Re: [openstack-dev] [Murano] Improvements to MuranoPL contracts
Hi Stan, I don't understand well your third problem/solution statement. Could you please give an example of the proposed behaviour vs the current one? The last solution - to have not-nullable primitive types by default - looks good to me, but it breaks the compatibility with the existing contracts. If some package developers have already got $.int() in their classes, we cannot be sure if this is a mistake or an intention to make the property nullable. In the latter case their code will be broke, as the null values will not be accepted anymore. I understand that currently the adoption of MuranoPL is quite low and most of its users are murano-contributors and so are probably reading this mailing list, so this breaking change should not surprise them, but I'd rather be careful with such things anyway. I suggest to discuss this today on a weekly meeting in IRC. If everybody are fine with such changes, then let's do them, as they are reasonable. -- Regards, Alexander Tivelkov On Mon, May 26, 2014 at 6:40 PM, Stan Lagun sla...@mirantis.com wrote: Hello everyone! Recently I've been looking for a best way to fix incorrect handling of defaults in MuranoPL's property contracts [1]. I analyzed how contracts used in applications that are currently in app-incubator, typical mistakes and usage patterns. I've found number of places where contract system can be significantly improved. Here is my list of proposed improvements I'd like to deliberate with you: Problem: when contract violation happens it is very hard to tell without debugger what went wrong. Especially this is a problem for complex nested structures. Currently it even impossible to tell which property caused exception during class load Solution: during contract traversal keep track of path to a part of contract being processed (for example 'foo/0/bar'). Include that path in every thrown exception alongside with human-readable description describing what value was processed and what contract was violated. Prepend property name to exception text so that is would be clear what property cannot be initialized. The same for method arguments. --- Problem: Single default value is not enough for properties that are not of scalar type. For example if we have contract like - name: $.string() disabled: $.bool() (array of structures) it is reasonable to have default value for disabled attribute of each list entry whereas there is no meaningful default for name attribute. Current approach allows to provide initial filling of entire array which is obviously not what developer expects. Solution: to have Default reflect contract structure so that different parts of Default can serve as an independent defaults for corresponding parts of the contract. Default need to be traversed in parallel with contract spec and provided value. Default value need not provide value for every single attribute mentioned in contract spec and thus can be simpler than contract itself. --- Problem: Default value is only used when no property value provided at all. Null (None) value is a valid value even if it not valid for particular contract. Thus is null is passed Default is not used. Solution: to treat every missing value as null. Missing Default is also considered to be null. This greatly simplifies understanding of contracts and client development (it is easier to set attribute value to None rather then deleting attribute from object if default value is desired, especially for generic clients) --- Problem: most of the time developer writes $.int() contract he doesn't realize that null is also valid value for such contract and the correct form is $.int().notNull(). This is even more obvious in case of bool(). Developers are lazy and usually forget to wright long verbose contracts. Solution: to make all primitive types be not-nullable with obvious defaults (0 for int, false for bool). This is enough for 95% of use cases. For the rest 5% where null value does valid to have separate contract methods optionalInt(), optionalBool() etc. As for strings I think that the same approach should be used - $.string() is a non-nullable sequence of chars with empty string as a default. And there should also be optionalString() that es equal to current $.string(). But in most cases when developer writes $.string() what he really wants is $.string().notNull().trim().notEmpty(). So while it is reasonable to have all this helper functions (notEmpty(), notBlank(), trim() etc.) is would be great to have one contract function that does exactly that. --- While some of proposed changes can possible break existing contracts in practice they just legalize current state of affairs so nothing would break. What do you think? [1]: https://bugs.launchpad.net/murano/+bug/1313694 Sincerely yours, Stan Lagun Principal Software Engineer @ Mirantis sla...@mirantis.com ___ OpenStack-dev
Re: [openstack-dev] [Murano] Improvements to MuranoPL contracts
Got it.
What about this it is easier to set attribute value to None rather then
deleting attribute from object if default value is desired, especially for
generic clients?
Do I understand correctly that setting some property to None should leads
to automatic assignment of the default value?
As for me, this looks not obvious, and I would prefer not to have such
implicit assignments
--
Regards,
Alexander Tivelkov
On Tue, May 27, 2014 at 4:05 PM, Stan Lagun sla...@mirantis.com wrote:
Hi Alex,
1. This is directly related to the bug mentioned above.
Taking an example from launchpad
networks:
Contract:
environment: $.bool().notNull()
flat: $.bool().notNull()
custom: [$.class(Network).notNull()]
Default:
environment: true
flat: false
currently if no value was passed for 'networks' property (and no value
means no such key, null value doesn't count) then default value would be
used ({environment: true, flat: false}) and contract would be applied
to that value. If you pass networks: {flat: true} you will get
exception as this value doesn't matched contract (for 'environment' key).
With new approach exactly the same Default is interpreted as a set of
independent default - default for 'environment' attribute and default for
'flat' attribute. So {flat: true} turns into {environment: true,
flat: true}. With proposed changes the example above can be rewritten as
networks:
Contract:
environment: $.bool()
flat: $.bool()
custom: [$.class(Network)]
Default:
environment: true
2. Yes, I do agree it breaks compatibility. I believe we will brake it
anyway because MuranoPL is still far from mature and it is very likely that
some new future would require incompatible change. Thats was one of the
purpose of app-incubator repository - if we make breaking changes we can
walk through all applications and fix them. I hope that all such changes
will be made in Juno time frame.
As for this particular change it is 1% chance that $.int() contract was
intentionally written to accept null and 99% that it was mistake. So making
$.int() not-nullable we do much more good than cause harm
Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis
sla...@mirantis.com
On Tue, May 27, 2014 at 1:37 PM, Alexander Tivelkov
ativel...@mirantis.com wrote:
Hi Stan,
I don't understand well your third problem/solution statement. Could you
please give an example of the proposed behaviour vs the current one?
The last solution - to have not-nullable primitive types by default -
looks good to me, but it breaks the compatibility with the existing
contracts. If some package developers have already got $.int() in their
classes, we cannot be sure if this is a mistake or an intention to make the
property nullable. In the latter case their code will be broke, as the null
values will not be accepted anymore.
I understand that currently the adoption of MuranoPL is quite low and
most of its users are murano-contributors and so are probably reading this
mailing list, so this breaking change should not surprise them, but I'd
rather be careful with such things anyway. I suggest to discuss this today
on a weekly meeting in IRC. If everybody are fine with such changes, then
let's do them, as they are reasonable.
--
Regards,
Alexander Tivelkov
On Mon, May 26, 2014 at 6:40 PM, Stan Lagun sla...@mirantis.com wrote:
Hello everyone!
Recently I've been looking for a best way to fix incorrect handling of
defaults in MuranoPL's property contracts [1]. I analyzed how contracts
used in applications that are currently in app-incubator, typical mistakes
and usage patterns. I've found number of places where contract system can
be significantly improved. Here is my list of proposed improvements I'd
like to deliberate with you:
Problem: when contract violation happens it is very hard to tell without
debugger what went wrong. Especially this is a problem for complex nested
structures. Currently it even impossible to tell which property caused
exception during class load
Solution: during contract traversal keep track of path to a part of
contract being processed (for example 'foo/0/bar'). Include that path in
every thrown exception alongside with human-readable description describing
what value was processed and what contract was violated. Prepend property
name to exception text so that is would be clear what property cannot be
initialized. The same for method arguments.
---
Problem: Single default value is not enough for properties that are not
of scalar type.
For example if we have contract like
- name: $.string()
disabled: $.bool()
(array of structures) it is reasonable to have default value for
disabled attribute of each list entry whereas there is no meaningful
default for name attribute. Current approach allows to provide initial
filling of entire array which is obviously not what
Re: [openstack-dev] [infra] Gerrit downtime on May 23 for project renames
Hi folks, After the murano-api repo was renamed to murano, all the old changesets in gerrit for this repository are missing in all the queries: https://review.openstack.org/#/q/project:stackforge/murano,n,z; returns only new (or recently updated) changesets, while https://review.openstack.org/#/q/project:stackforge/murano-api,n,z; does not return anything at all. Even the currently open (i.e. not merged) changesets are not shown. Direct links to the changesets still work, and once rebased to a new repo the changeset becomes visible, however all the history seems lost. Previously, if I was looking for the reasons behind some line of code, I could git-annotate the code, find the commit message and use the gerrit change-id from that message to locate the change in gerrit. Not it seems impossible. Is there any way to recover the old changesets? -- Regards, Alexander Tivelkov On Sat, May 24, 2014 at 2:58 AM, James E. Blair jebl...@openstack.orgwrote: jebl...@openstack.org (James E. Blair) writes: Hi, On Friday, May 23 at 21:00 UTC Gerrit will be unavailable for about 20 minutes while we rename some projects. Existing reviews, project watches, etc, should all be carried over. This is complete. The actual list of renamed projects is: stackforge/barbican - openstack/barbican openstack/oslo.test - openstack/oslotest openstack-dev/openstack-qa - openstack-attic/openstack-qa openstack/melange - openstack-attic/melange openstack/python-melangeclient - openstack-attic/python-melangeclient openstack/openstack-chef - openstack-attic/openstack-chef openstack/database-api - openstack-attic/database-api stackforge/climate - stackforge/blazar stackforge/climate-nova - stackforge/blazar-nova stackforge/python-climateclient - stackforge/python-blazarclient stackforge/murano-api - stackforge/murano openstack/baremetal-specs - openstack/ironic-specs openstack/object-specs - openstack/swift-specs openstack/orchestration-specs - openstack/heat-specs openstack/identity-specs - openstack/keystone-specs openstack/telemetry-specs - openstack/ceilometer-specs The consensus about specs repo names seemed to be converging on 'codename' significantly enough that we felt it would be best to drop the previous plan to rename to program name, and instead rename the handful of brand-new repos that had been created to their codenames. Giving that kind of notice isn't our favorite thing to do, to put it mildly, but I think this is for the best and we can get back to actually writing specs now. And maybe code. Thanks to Sergey, Monty, Clark and Jeremy all of whom did a great deal of work on this. -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance] Call for alternate session leader for Signed Images
This is an interesting topic indeed: we though about the very same idea in Murano (checking the origin of app definitions is a good thing for an application catalog). If there are similar ideas in other projects, then this definitely should belong to Glance catalog. -- Regards, Alexander Tivelkov On Thu, May 8, 2014 at 11:36 AM, Flavio Percoco fla...@redhat.com wrote: On 07/05/14 22:45 -0700, Mark Washenberger wrote: Hi folks, Unfortunately, the leader for one of our proposed sessions is now unable to attend the summit. The topic in question is Signed Images [1] and was allocated a half-session slot. This is a call out to see if there are any other folks who would like to lead this discussion. If not, no big deal. We will have other things to discuss during that time. This seems a quite requested feature and an interesting topic. I'll like to lead this session. I'll sync w/ the former leader of this session and prepare one based on his ideas. Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Introducing Project Graffiti
Hi Travis, It will be great to have your team join the artifact repository initiative. We are going to have a session at the summit ([1]) to discuss the design, and you most welcome to join. We'll publish the etherpad with brief agenda for this session later this week. Speaking about Murano you've got a very good point: currently Murano's application catalog is very tightly coupled with the workflow engine. It happens because Murano aims not only to provide the pure catalog capabilities, but also to manage the lifecycle of the deployed applications: provide ways to handle external and internal events, maintain the applications' state, report usage statistics etc - all of this usually requires workflows, and these workflows have to be aware about the applications' data structure and interdependencies. That's where the idea of Murano's workflow engine came from. However, it turned out to be overcomplicated and having some potential overlaps with existing OpenStack projects (you probably saw a very long discussions here about it) - and we are now working hard to fix this, by establishing a clean separation between plain catalog functionality of Murano and the workflow-related processes: some of them should go to different projects, some will remain in Murano but in more unified and clear manner. We'll have a couple of design sessions on this as well - at the cross-project workshop ([2], [3]) - feel free to join as well, if you are interested. Speaking about the data storage for Murano, currently we look forward to the Glance Artifact Repository: we expect murano application packages (being composed out of Heat templates, workflows, binary files and other stuff) to be stored as composite multi-part artifacts in Glance. Each part of this artifact will be an entry-point for some action is some OpenStack service (like you said: boot sources for Nova, templates for Heat etc), while the workflow which coordinates this various services and tells them which part of artifact to boot from will be run as the higher-level layer service. Hope this answer your questions. If you have more to ask, please feel free to join our weekly meeting in IRC ([4]) - the nearest one is today (Tuesday), at 17:00 UTC Thanks! [1] http://junodesignsummit.sched.org/event/b0e1f0cbefffa942e276f1add9f85d03#.U2h8JF6nDWU [2] http://junodesignsummit.sched.org/event/b7f067ff055ff7db9c92867f3febe1d9#.U2iAd16nDWU [3] http://junodesignsummit.sched.org/event/556d10915a1a0a2f1aee3ba286826b60#.U2iAeF6nDWU [4] https://wiki.openstack.org/wiki/Meetings/MuranoAgenda -- Regards, Alexander Tivelkov On Tue, May 6, 2014 at 9:37 AM, Tripp, Travis S travis.tr...@hp.com wrote: Hi Angus, This seems neat, but also seems to have some overlap with glance's new catalog and some of the things Murano are doing. Have you had a look at those efforts? Thanks! We have been keeping an eye on the Glance work and the Murano work, and your email reminded me to catch back up on both of them. I think in both cases that the Graffiti concepts are complementary. However, strictly speaking from a bird's eye view on application categorization, there is some reconciliation to work out on that aspect. Regarding the Glance artifact repository, this looks like a nice revamp of its concepts. Most of it seems to be very related to handling artifacts, dependencies, versions, and relationships associated with packaging in a generic way so that external services can use it in a variety of ways. There is one feature that was discussed in the last Glance meeting logs that I think we might be able to leverage for some of the Graffiti concepts. That is a dynamic schemas API. Perhaps, we can build the Graffiti dictionary concepts on top of it. We're definitely interested in anything that creates less moving parts for us and is already part of the standard OpenStack ecosystem. For Murano, the pure application catalog UI is an interesting concept that today still seems to be intertwined with the Murano workflow engine. It isn't clear to me if the intent is for it to eventually become the UI for everything application related, including Solum and pure Heat templates? From the mailing list discussions, it seems that this is still a rather unresolved question. For us we'd like to be able to provide end user help with even the existing launch instance UI. Also, one of the goals of the Graffiti concepts is to be able to directly tag resources with metadata that comes from multiple sources, whether that is system provided (e.g. various compute capabilities) or user provided (e.g. categories or software tags) and then be able to search on them. This can be used for boot sources, flavors, host aggregates, etc, and perhaps even networks in the future. It seems possible that Murano may just be a consumer of some of the same data? -Travis -Original Message- From: Angus Salkeld [mailto:angus.salk...@rackspace.com] Sent
Re: [openstack-dev] [Heat] [Glance] How about managing heat template like flavors in nova?
Hi Randall, The current design document on artifacts in glance is available here [1]. It was just published, we are currently gathering the feedback on it. Please feel free to add comments to the document or write any suggestions or questions to the ML. There was a little discussion on yesterdays IRC meeting ([2]), I've answered some questions there. I will be happy to answer any questions directly (I am ativelkov in IRC) or we may discuss the topic in more details on the next Glance meeting next Thursday. Looking forward for collaboration with Heat team on this topic. [1] https://docs.google.com/document/d/1tOTsIytVWtXGUaT2Ia4V5PWq4CiTfZPDn6rpRm5In7U/edit?usp=sharing [2] http://eavesdrop.openstack.org/meetings/glance/2014/glance.2014-04-24-14.01.log.html -- Regards, Alexander Tivelkov On Mon, Apr 21, 2014 at 4:29 PM, Randall Burt randall.b...@rackspace.com wrote: We discussed this with the Glance community back in January and it was agreed that we should extend Glance's scope to include Heat templates as well as other artifacts. I'm planning on submitting some patches around this during Juno. Adding the Glance tag as this is relevant to them as well. Original message From: Mike Spreitzer Date:04/19/2014 9:43 PM (GMT-06:00) To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Heat] How about managing heat template like flavors in nova? Gouzongmei gouzong...@huawei.com wrote on 04/19/2014 10:37:02 PM: We can supply APIs for getting, putting, adding and deleting current templates in the system, then when creating heat stacks, we just need to specify the name of the template. Look for past discussion of Heat Template Repository (Heater). Here is part of it: https://wiki.openstack.org/wiki/Heat/htr Regards, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Glance] Artifact Repository Design
Hi stackers, As agreed, I was working on designing the proposal on the Artifact Repository for Glance. It took me longer then expected - sorry about that - but finally I have a complete (hopefully) document, describing all the details about artifacts and their lifecycle. I've put the document into google docs ([1]) rather then etherpad, as its quite long and has some tables and formatting, while the google docs have an ability to add comments and highlights which will help discussing it. I hope that 11 pages is not too much for the starting point. I'd like to begin the discussion (or at least to introduce the doc) at today's meeting. I've put the item into the agenda ([2]) [1] https://docs.google.com/document/d/1tOTsIytVWtXGUaT2Ia4V5PWq4CiTfZPDn6rpRm5In7U/edit?usp=sharing [2] http://etherpad.openstack.org/p/glance-team-meeting-agenda -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [murano] Proposal to add Ruslan Kamaldinov to murano-core team
+1 Totally agree -- Regards, Alexander Tivelkov On Thu, Apr 17, 2014 at 4:37 PM, Timur Sufiev tsuf...@mirantis.com wrote: Guys, Ruslan Kamaldinov has been doing a lot of things for Murano recently (including devstack integration, automation scripts, making Murano more compliant with OpenStack standards and doing many reviews). He's actively participating in our ML discussions as well. I suggest to add him to the core team. Murano folks, please say your +1/-1 word. -- Timur Sufiev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Separating our Murano PL core in own package
Hi, I suggest to move all needed Powershell scripts and etc. to the main repository 'murano' in the separate folder. +1 on this. The scripts will not go inside the PyPi package, they will be just grouped in a subfolder. Completely agree on the repo-reorganization topic in general. However And I personally will do everything to prevent creation of new repo for Murano. Well, this may be unavoidable :) We may face a need to create a murano-contrib repository where Murano users will be able to contribute sources of their own murano packages, improve the core library etc. Given that we get rid of murano-conductor, murano-repository, murano-metadataclient, murano-common, murano-tests and, probably, murano-deployment, we are probably ok with having one more. Technically, we may reuse murano-repository for this. But this can be discussed right after there 0.5 release. -- Regards, Alexander Tivelkov On Tue, Mar 25, 2014 at 12:09 PM, Timur Nurlygayanov tnurlygaya...@mirantis.com wrote: Dmitry, I suggest to move all needed Powershell scripts and etc. to the main repository 'murano' in the separate folder. On Tue, Mar 25, 2014 at 11:38 AM, Dmitry Teselkin dtesel...@mirantis.comwrote: Ruslan, What about murano-deployment repo? The most important part of it are PowerSheel scripts, Windows Image Builder, package manifests, and some other scripts that better to keep somewhere. Where do we plan to move them? On Mon, Mar 24, 2014 at 10:29 PM, Ruslan Kamaldinov rkamaldi...@mirantis.com wrote: On Mon, Mar 24, 2014 at 10:08 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: Seeing that the following repos already exist, maybe there is some need for cleanup? - https://github.com/stackforge/murano-agent - https://github.com/stackforge/murano-api - https://github.com/stackforge/murano-common - https://github.com/stackforge/murano-conductor - https://github.com/stackforge/murano-dashboard - https://github.com/stackforge/murano-deployment - https://github.com/stackforge/murano-docs - https://github.com/stackforge/murano-metadataclient - https://github.com/stackforge/murano-repository - https://github.com/stackforge/murano-tests ...(did I miss others?) Can we maybe not have more git repositories and instead figure out a way to have 1 repository (perhaps with submodules?) ;-) It appears like murano is already exploding all over stackforge which makes it hard to understand why yet another repo is needed. I understand why from a code point of view, but it doesn't seem right from a code organization point of view to continue adding repos. It seems like murano (https://github.com/stackforge/murano) should just have 1 repo, with sub-repos (tests, docs, api, agent...) for its own organizational usage instead of X repos that expose others to murano's internal organizational details. -Josh Joshua, I agree that this huge number of repositories is confusing for newcomers. I've spent some time to understand mission of each of these repos. That's why we already did the cleanup :) [0] And I personally will do everything to prevent creation of new repo for Murano. Here is the list of repositories targeted for the next Murano release (Apr 17): * murano-api * murano-agent * python-muranoclient * murano-dashboard * murano-docs The rest of these repos will be deprecated right after the release. Also we will rename murano-api to just murano. murano-api will include all the Murano services, functionaltests for Tempest, Devstack scripts, developer docs. I guess we already can update README files in deprecated repos to avoid further confusion. I wouldn't agree that there should be just one repo. Almost every OpenStack project has it's own repo for python client. All the user docs are kept in a separate repo. Guest agent code should live in it's own repository to keep number of dependencies as low as possible. I'd say there should be required/comfortable minimum of repositories per project. And one more nit correction: OpenStack has it's own git repository [1]. We shoul avoid referring to github since it's just a convinient mirror, while [1] is an official OpenStack repository. [0] https://blueprints.launchpad.net/murano/+spec/repository-reorganization [1] http://git.openstack.org/cgit/ Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Dmitry Teselkin Deployment Engineer Mirantis http://www.mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Timur, QA Engineer OpenStack Projects Mirantis Inc ___ OpenStack-dev mailing list OpenStack-dev
Re: [openstack-dev] Separating our Murano PL core in own package
Hi Serg, Are you proposing to have a standalone git repository / stack forge project for that? Or just a separate package inside our primary murano repo? -- Regards, Alexander Tivelkov On Mon, Mar 24, 2014 at 12:00 PM, Serg Melikyan smelik...@mirantis.comwrote: Programming Language, AFAIK On Mon, Mar 24, 2014 at 11:46 AM, Oleg Gelbukh ogelb...@mirantis.comwrote: What does PL stand for, anyway? -- Best regards, Oleg Gelbukh On Mon, Mar 24, 2014 at 11:39 AM, Serg Melikyan smelik...@mirantis.comwrote: because 'dsl'/'language' terms are too broad. Too broad in general, but we choose name for sub-package, and in murano term 'language' mean Murano PL. +1 for language On Mon, Mar 24, 2014 at 11:26 AM, Timur Sufiev tsuf...@mirantis.comwrote: +1 for muranoapi.engine.murano_pl, because 'dsl'/'language' terms are too broad. On Mon, Mar 24, 2014 at 12:48 AM, Timur Nurlygayanov tnurlygaya...@mirantis.com wrote: Hi Serg, This idea sounds good, I suggest to use name 'murano.engine.murano_pl' (not just common name like 'language' or 'dsl', but name, which will be based on 'MuranoPL') Do we plan to support the ability to define different languages for Murano Engine? Thank you! On Sun, Mar 23, 2014 at 1:05 PM, Serg Melikyan smelik...@mirantis.com wrote: There is a idea to separate core of Murano PL implementation from engine specific code, like it was done in PoC. When this two things are separated in different packages, we will be able to track and maintain our language core as clean as possible from engine specific code. This will give to us an ability to easily separate our language implementation to a library. Questions is under what name we should place core of Murano PL? 1) muranoapi.engine.language; 2) muranoapi.engine.dsl; 3) suggestions? -- Serg Melikyan, Senior Software Engineer at Mirantis, Inc. http://mirantis.com | smelik...@mirantis.com +7 (495) 640-4904, 0261 +7 (903) 156-0836 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Timur, QA Engineer OpenStack Projects Mirantis Inc ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Timur Sufiev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Serg Melikyan, Senior Software Engineer at Mirantis, Inc. http://mirantis.com | smelik...@mirantis.com +7 (495) 640-4904, 0261 +7 (903) 156-0836 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Serg Melikyan, Senior Software Engineer at Mirantis, Inc. http://mirantis.com | smelik...@mirantis.com +7 (495) 640-4904, 0261 +7 (903) 156-0836 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano][Heat] MuranoPL questions?
Hi, So that's where I want to make a first stop. If your primary user is not a developer, there is no reason to introduce a DSL for security reasons. The provider can trust the code he writes, and there is no need to create a dedicated language. I thinks this need to be clarified. The provider does not write code: the provider just hosts the cloud, acting as a moderator. The code is written by another category of end-users, called Application Publishers. This category is untrusted - that's the nature of Application Catalog: anybody can upload everything. The publishers (who write DSL) should not be confused with users (who define object models using it). These are different roles. -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] New API methods for App Catalog UI
Hi Georgy, There was already a discussion of these APIs [1] about some time ago, the draft for API has been proposed here [2], the etherpad for discussion and feedback was created [3] and the direction was already approved in the blueprint [4]. As far as I know, the work on this set of APIs has already begun. Please align your vision with this spec. We may discuss it today on the weekly meeting in IRC. [1] http://lists.openstack.org/pipermail/openstack-dev/2014-March/028886.html [2] http://docs.muranorepositoryapi.apiary.io [3] https://etherpad.openstack.org/p/muranorepository-api [4] https://blueprints.launchpad.net/murano/+spec/murano-repository-api-v2 -- Regards, Alexander Tivelkov On Mon, Mar 10, 2014 at 7:21 PM, Georgy Okrokvertskhov gokrokvertsk...@mirantis.com wrote: Hi, Murano is moving towards App Catalog functionality and in order to support this new aspect in the UI we need to add new API methods to cover App Catalog operations. Currently the vision for App Catalog API is the following: 1) All App create operations will be covered by metadata repository API which will eventually be a part of Glance Artifacts functionality. New application creation will be technically a creation of a new artifact and uploading it to metadata repository. The sharing and distribution aspects will be covered by the same artifact repository functionality. 2) App Listing and App Catalog rendering will be covered by a new Murano API. The reason for that is to keep UI thin and keep package representation aspects out of the general artifacts repository. The list of new API functions is available here: https://etherpad.openstack.org/p/MuranoAppCatalogAPI This is a first draft to cover minimal UI rendering requirements. Thanks Georgy -- Georgy Okrokvertskhov Architect, OpenStack Platform Products, Mirantis http://www.mirantis.com Tel. +1 650 963 9828 Mob. +1 650 996 3284 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Community Meeting reminder - 03/04/2014
Hi folks, This is just a reminder that we are going to have a regular weekly meeting in IRC (#openstack-meeting-alt at freenode) today at 17:00 UTC (9am PST). The agenda is available at https://wiki.openstack.org/wiki/Meetings/MuranoAgenda#Agenda Feel free to add anything you want to discuss to the Agenda. See you there! -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Community Meeting minutes - 03/04/2014
Hi, Thanks for joining murano weekly meeting. Here are the meeting minutes and the logs: http://eavesdrop.openstack.org/meetings/murano/2014/murano.2014-03-04-17.01.html http://eavesdrop.openstack.org/meetings/murano/2014/murano.2014-03-04-17.01.log.html See you next week! -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Hooking the external events discussion
Hi folks, On today's IRC meeting there was a very interesting discussion about publishing of handlers for external events in Murano applications. It turns out that the topic is pretty hot and requires some more discussions. So, it was suggested to host an additional meeting to cover this topic. So, let's meet tomorrow at #murano channel on freenode. The suggested time is 16:00 UTC (8am PST). Anybody who is interested in the topic, please feel free to join! Thanks -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat]Heat use as a standalone component for Cloud Managment over multi IAAS
Hi Charles, If you are looking for the analogues of Juju in OpenStack, you probably may take a look at Murano Project [1]. It is an application catalog backed with a powerful workflow execution engine, which is built on top of Heat's orchestration, but run's at a higher level. It has borrowed lots of idea from Juju (or, more precisely, both took a lot from Amazon's OpsWorks ideas). Also, if you are looking to orchestrate on top of non-openstack clouds -- Regards, Alexander Tivelkov [1[ - https://wiki.openstack.org/wiki/Murano On Wed, Feb 26, 2014 at 5:47 PM, Charles Walker charles.walker...@gmail.com wrote: Hi, I am trying to deploy the proprietary application made in my company on the cloud. The pre requisite for this is to have a IAAS which can be either a public cloud or private cloud (openstack is an option for a private IAAS). The first prototype I made was based on a homemade python orchestrator and apache libCloud to interact with IAAS (AWS and Rackspace and GCE). The orchestrator part is a python code reading a template file which contains the info needed to deploy my application. This template file indicates the number of VM and the scripts associated to each VM type to install it. Now I was trying to have a look on existing open source tool to do the orchestration part. I find JUJU (https://juju.ubuntu.com/) or HEAT ( https://wiki.openstack.org/wiki/Heat). I am investigating deeper HEAT and also had a look on https://wiki.openstack.org/wiki/Heat/DSL which mentioned: *Cloud Service Provider* - A service entity offering hosted cloud services on OpenStack or another cloud technology. Also known as a Vendor. I think HEAT as its actual version will not match my requirement but I have the feeling that it is going to evolve and could cover my needs. I would like to know if it would be possible to use HEAT as a standalone component in the future (without Nova and other Ostack modules)? The goal would be to deploy an application from a template file on multiple cloud service (like AWS, GCE). Any feedback from people working on HEAT could help me. Thanks, Charles. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat]Heat use as a standalone component for Cloud Managment over multi IAAS
Hi Charles, If you are looking for the analogues of Juju in OpenStack, you probably may take a look at Murano Project [1]. It is an application catalog backed with a powerful workflow execution engine, which is built on top of Heat's orchestration, but run's at a higher level. It has borrowed lots of idea from Juju (or, more precisely, both took a lot from Amazon's OpsWorks ideas). Also, if you are looking to orchestrate on top of non-openstack clouds, then Murano's DSL may also be an answer: Murano's workflows may be designed to trigger any external APIs, not necessary OpenStack-only, so the technical possibility to orchestrate AWS and GCE exists in Murano's design, yet not present in the current roadmap. Please feel free to ask for more details either in [Murano] ML or at #murano channel at Freenode. Thanks [1] - https://wiki.openstack.org/wiki/Murano -- Regards, Alexander Tivelkov -- Regards, Alexander Tivelkov On Wed, Feb 26, 2014 at 5:47 PM, Charles Walker charles.walker...@gmail.com wrote: Hi, I am trying to deploy the proprietary application made in my company on the cloud. The pre requisite for this is to have a IAAS which can be either a public cloud or private cloud (openstack is an option for a private IAAS). The first prototype I made was based on a homemade python orchestrator and apache libCloud to interact with IAAS (AWS and Rackspace and GCE). The orchestrator part is a python code reading a template file which contains the info needed to deploy my application. This template file indicates the number of VM and the scripts associated to each VM type to install it. Now I was trying to have a look on existing open source tool to do the orchestration part. I find JUJU (https://juju.ubuntu.com/) or HEAT ( https://wiki.openstack.org/wiki/Heat). I am investigating deeper HEAT and also had a look on https://wiki.openstack.org/wiki/Heat/DSL which mentioned: *Cloud Service Provider* - A service entity offering hosted cloud services on OpenStack or another cloud technology. Also known as a Vendor. I think HEAT as its actual version will not match my requirement but I have the feeling that it is going to evolve and could cover my needs. I would like to know if it would be possible to use HEAT as a standalone component in the future (without Nova and other Ostack modules)? The goal would be to deploy an application from a template file on multiple cloud service (like AWS, GCE). Any feedback from people working on HEAT could help me. Thanks, Charles. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Object-oriented approach for defining Murano Applications
Hi Keith, The question with Heat plugins is a good one. It actually has several different answers. The most important one is that the actual implementation of this approach may be considered as out of scope for Murano. Think of Murano as an application integration engine, which allows existing application to communicate with each other and with the openstack infrastructure via multiple interfaces. The possibility of this interaction does matter a lot, while the actual implementation of the interfaces is much less important. Application Publishers should be able to execute something like 'instance.takeSnapshot()' or 'databaseInstance.Migrate(anotherDatabaseInstance)' from the DSL-code of their workflows, while the actual implementation of these actions may be different for different classes. Murano will definitely come bundled with the support of Heat templates as its primary OpenStack interface, so, if an application publisher decides to execute some specific action via a custom heat plugin, Murano will provide all means to use this plugin out-of-the-box. So, in general, the plugin-approach is perfectly fine, it already fits into Murano, and that is intentional: our ultimate goal is to integrate Murano into existing OpenStack infrastructure as much as possible. The final desicion which approach to choose is up to the publisher of the specific application though. However, in some cases relying on the plugins seems to be a bad idea. When you create a custom plugin for a custom task you have to install the plugin to the Heat at the target cloud first. And this contradicts with the whole purpose of application catalog, where all the needed dependencies of the application are placed into its package by the publisher, and once the package is put into the catalog it is immediately available for usage. We cannot allow to put plugins into the packages, as plugins are actually written in python, while allowing to upload and run an arbitrary python code to untrusted users is definitely a security breach. So, plugins are ok for the most common and popular tasks (but in this case these types of resources should be eventually placed into the Heat itself, right?), but may be a bad choice for custom tasks. And the last but not least, the thing which Georgy has already mentioned: I personally feel a bit uncomfortable when trying to think about the processes and actions as objects persisted in a Heat stack. These are different kinds of entities, and in my opinion events are dynamic and transient, while the Heat stack is stable and persisted. Events can modify the stack - and that is why we need the workflows - but they should not be persisted on their own. I don't know how to express it in a more formal way, and that is just my gut feeling which definitely has to be discussed more. I am sure we will come back to this topic when the DSL is finished and we started implementing the bases class library for Murano and its external interfaces with Heat and other services. Thanks! -- Regards, from Alexander Tivelkov On Tue, Feb 25, 2014 at 1:44 AM, Keith Bray keith.b...@rackspace.comwrote: Have you considered writing Heat resource plug-ins that perform (or configure within other services) instance snapshots, backups, or whatever other maintenance workflow possibilities you want that don't exist? Then these maintenance workflows you mention could be expressed in the Heat template forming a single place for the application architecture definition, including defining the configuration for services that need to be application aware throughout the application's life . As you describe things in Murano, I interpret that you are layering application architecture specific information and workflows into a DSL in a layer above Heat, which means information pertinent to the application as an ongoing concern would be disjoint. Fragmenting the necessary information to wholly define an infrastructure/application architecture could make it difficult to share the application and modify the application stack. I would be interested in a library that allows for composing Heat templates from snippets or fragments of pre-written Heat DSL... The library's job could be to ensure that the snippets, when combined, create a valid Heat template free from conflict amongst resources, parameters, and outputs. The interaction with the library, I think, would belong in Horizon, and the Application Catalog and/or Snippets Catalog could be implemented within Glance. Also, there may be workflow steps which are not covered by Heat by design. For example, application publisher may include creating instance snapshots, data migrations, backups etc into the deployment or maintenance workflows. I don't see how these may be done by Heat, while Murano should definitely support this scenarios. From: Alexander Tivelkov ativel...@mirantis.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack
[openstack-dev] [Murano] Community meeting reminder - 02/25/2014
Hi folks, This is just a reminder that we are going to have a regular weekly meeting in IRC today at 17:00 UTC (9am PST). The agenda is available at https://wiki.openstack.org/wiki/Meetings/MuranoAgenda#Agenda This may be a good chance to discuss all the recent questions related to Murano DSL and object-oriented approach, review the current status of our incubation request and so on. As usual, please feel free to add your own agenda items. -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Glance][Artifacts] Artifact dependencies: Strict vs Soft
Hi folks, While I am still working on designing artifact-related APIs (sorry, the task is taking me longer then expected due to a heavy load in Murano, related to the preparation of incubation request), I've got a topic I wanted to discuss with the broader audience. It seems like we have agreed on the idea that the artifact storage should support dependencies between the artifacts: ability for any given artifact to reference some other artifacts as its dependencies, and the API call which will allow to retrieve all the dependency graph of the given artifact (i.e. its direct and transitive dependecies) Another idea which was always kept in mind when we were designing the artifact concept was artifact versioning: the system should allow to store different artifact having the identical name but different versions, and the API should be able to return the latest (based on some notation) version of the artifact. Being able to construct such a queries actually gives an ability to define kind of aliases, so the url like /v2/artifacts?type=imagename=ubuntuversion=latest will always return the latest version of the given artifact (ubuntu image in this case). The need to be able to define such aliaces was expressed in [1], and the ability to satisfy this need with artifact API was mentioned at [2] But combining these two ideas brings up an interesting question: how should artifacts define their dependencies? Should this be an explicit strict reference (i.e. referencing the specific artifact by its id), or it should be an implicit soft reference, similar to the alias described above (i.e. specifying the dependency as A requires the latest version of B or even A requires 0.2=B0.3)? The later seems familiar: it is similar to pip dependency specification, right? This approach obviosuly may be very usefull (at least I clearly see its benefits for Murano's application packages), but it implies lazy evaluation, which may dramatically impact the performance. In contrary, the former approach - with explicit references - requires much less computation. Even more, if we decide that the artifact dependencies are immutable, this will allow us to denormalize the storage of the dependency graph and store all the transitive dependencies of the given artifact in a flat table, so the dependency graph may be returned by a sinle SQL query, without a need for recursive calls, which are otherwise unavoidable in the normalized database storing such hierarchical structures. Meanwhile, the mutability of dependencis is also unclear to me: ability to modify them seems to have its own pros and cons, so this is another topic to dicsuss. I'd like to hear your opinion on all of these. Any feedback is welcome, and we may come back to this topic on the Thursday's meeting. Thanks! [1] https://blueprints.launchpad.net/glance/+spec/glance-image-aliases [2] https://blueprints.launchpad.net/glance/+spec/artifact-repository-api -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Community meeting minutes - 02/25/2014
Hi, Thanks for joining murano weekly meeting. Here are the meeting minutes and the logs: http://eavesdrop.openstack.org/meetings/murano/2014/murano.2014-02-25-17.00.html http://eavesdrop.openstack.org/meetings/murano/2014/murano.2014-02-25-17.00.log.html See you next week! -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Need a new DSL for Murano
Hi Dmitry, I agree with you on this vision, however we have to think more on the terminology: Service Catalog in OpenStack relates to Keystone (where by Service we mean Openstack's infrastructure-level services). I understand your concerns on runtime lifecycle vs code-to-binary lifecycle though - it is absolutely valid, and we do not want to have any overlap with Solum in this matter. -- Regards, Alexander Tivelkov On Mon, Feb 24, 2014 at 1:47 PM, Dmitry mey...@gmail.com wrote: I think this is the great new service which will accompany OpenStack Solum similarly as Bosh is accompanying Cloud Foundry and CloudForms is accompanying OpenShift. I wouldn't call it the Application Catalog but the Service Catalog, because of the primary focus on the Service Life-cycle management (in opposite to application lifecycle management which is focused on code-to-binary, execution, remote debugging and log grabbing etc). In order to make Murano the real Service Catalog, it should support (over DSL) run-time events processing such as service scaling (up/out/in/down), healing, replication, live-migration etc. In addition, it should support a template creation which could be used by Solum similar to Heroku BuildPack. I would happy to hear your opinion on how you envision the Murano's roadmap. Thanks, Dmitry ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Object-oriented approach for defining Murano Applications
Hi, I would like to initiate one more discussion about an approach we selected to solve a particular problem in Murano. The problem statement is the following: We have multiple entities like low level resources and high level application definitions. Each entity does some specific actions for example to create a VM or deploy application configuration. We want each entity's workflow code reusable in order to simplify development for a new application as the current approach with XML based rules requires significant efforts. After internal discussions inside Murano team we come up to the solution which uses a well known programmatic concept - classes, their inheritance and composition. In this thread I would like to share our ideas and discuss the implementation details. We want to represent each and every entity being manipulated by Murano, as an instance of some class. These classes will define structure of the entities and their behavior. Different entities may be combined together, interacting with each other, forming a composite environment. The inheritance may be used to extract common structure and functionality into generic superclasses, while having their subclasses to define only their specific attributes and actions. This approach is better to explain on some example. Let's consider the Active Directory windows service. This is one of the currently present Murano Applications, and its structure and deployment workflow is pretty complex. Let's see how it may be simplified by using the proposed object-oriented approach. First, let's just describe an Active Directory service in plain English. Active Directory service consists of several Controllers: exactly one Primary Domain Controller and, optionally, several Secondary Domain Controllers. Controllers (both primary and Secondary) are special Windows Instances, having an active directory server role activated. Their specific difference is in the configuration scripts which are executed on them after the roles are activated. Also, Secondary Domain Controllers have an ability to join to a domain, while Primary Domain Controller cannot do it. Windows Instances are regular machines having some limitations on the their images (it should, obviously, be Windows image) and hardware flavor (windows is usually demanding on resources). Also, windows machines may have some specific operations, like configuring windows firewall rules or defining local administrator password. And the machines in general (both Windows and any others) are simple entities which know how to create virtual machines in OpenStack clouds. Now, let's map this model to object-oriented concept. We get the following classes: 1. Instance. Defines common properties of virtual machines (flavor, image, hostname) and deployment workflow which executes a HEAT template to create an instance in the cloud. 2. WindowsInstance - inherits Instance. Defines local administrator account password and extends base deployment workflow to set this password and configure windows firewall - after the instance is deployed. 3. DomainMember - inherits Windows instance, defines a machine which can join an Active Directory. Adds a join domain workflow step 4. DomainController - inherits Windows instance, adds an Install AD Role workflow steps and extends the Deploy step to call it. 5. PrimaryController - inherits DomainContoller, adds a Configure as Primary DC workflow step and extends Deploy step to call it. Also adds a domainIpAddress property which is set during the deployment. 6. SecondaryController, inherits both DomainMember and DomainController. Adds a Configure as Secondary DC worflow step and extends Deploy() step to call it and the join domain step inherited from the Domain Member class. 7. ActiveDirectory - a primary class which defines an Active Directory application. Defines properties for PrimaryController and SecondaryControllers and a Deploy workflow which call appropriate workflows on the controllers. The simplified class diagram may look like this: So, this approach allows to decompose the AD deployment workflow into simple isolated parts, explicitly manage the state and create reusable entities (of course classes like Instance, WindowsInstance, DomainMember may be used by other Murano Applications). For me this looks much, much better than the current implicit state machine which we run based on XML rules. What do you think about this approach, folks? Do you think it will be easily understood by application developers? Will it be easy to write workflows this way? Do you see any drawbacks here? Waiting for your feedback. -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Object-oriented approach for defining Murano Applications
Sorry folks, I didn't put the proper image url. Here it is: https://creately.com/diagram/hrxk86gv2/kvbckU5hne8C0r0sofJDdtYgxc%3D -- Regards, Alexander Tivelkov On Mon, Feb 24, 2014 at 7:39 PM, Alexander Tivelkov ativel...@mirantis.comwrote: Hi, I would like to initiate one more discussion about an approach we selected to solve a particular problem in Murano. The problem statement is the following: We have multiple entities like low level resources and high level application definitions. Each entity does some specific actions for example to create a VM or deploy application configuration. We want each entity's workflow code reusable in order to simplify development for a new application as the current approach with XML based rules requires significant efforts. After internal discussions inside Murano team we come up to the solution which uses a well known programmatic concept - classes, their inheritance and composition. In this thread I would like to share our ideas and discuss the implementation details. We want to represent each and every entity being manipulated by Murano, as an instance of some class. These classes will define structure of the entities and their behavior. Different entities may be combined together, interacting with each other, forming a composite environment. The inheritance may be used to extract common structure and functionality into generic superclasses, while having their subclasses to define only their specific attributes and actions. This approach is better to explain on some example. Let's consider the Active Directory windows service. This is one of the currently present Murano Applications, and its structure and deployment workflow is pretty complex. Let's see how it may be simplified by using the proposed object-oriented approach. First, let's just describe an Active Directory service in plain English. Active Directory service consists of several Controllers: exactly one Primary Domain Controller and, optionally, several Secondary Domain Controllers. Controllers (both primary and Secondary) are special Windows Instances, having an active directory server role activated. Their specific difference is in the configuration scripts which are executed on them after the roles are activated. Also, Secondary Domain Controllers have an ability to join to a domain, while Primary Domain Controller cannot do it. Windows Instances are regular machines having some limitations on the their images (it should, obviously, be Windows image) and hardware flavor (windows is usually demanding on resources). Also, windows machines may have some specific operations, like configuring windows firewall rules or defining local administrator password. And the machines in general (both Windows and any others) are simple entities which know how to create virtual machines in OpenStack clouds. Now, let's map this model to object-oriented concept. We get the following classes: 1. Instance. Defines common properties of virtual machines (flavor, image, hostname) and deployment workflow which executes a HEAT template to create an instance in the cloud. 2. WindowsInstance - inherits Instance. Defines local administrator account password and extends base deployment workflow to set this password and configure windows firewall - after the instance is deployed. 3. DomainMember - inherits Windows instance, defines a machine which can join an Active Directory. Adds a join domain workflow step 4. DomainController - inherits Windows instance, adds an Install AD Role workflow steps and extends the Deploy step to call it. 5. PrimaryController - inherits DomainContoller, adds a Configure as Primary DC workflow step and extends Deploy step to call it. Also adds a domainIpAddress property which is set during the deployment. 6. SecondaryController, inherits both DomainMember and DomainController. Adds a Configure as Secondary DC worflow step and extends Deploy() step to call it and the join domain step inherited from the Domain Member class. 7. ActiveDirectory - a primary class which defines an Active Directory application. Defines properties for PrimaryController and SecondaryControllers and a Deploy workflow which call appropriate workflows on the controllers. The simplified class diagram may look like this: So, this approach allows to decompose the AD deployment workflow into simple isolated parts, explicitly manage the state and create reusable entities (of course classes like Instance, WindowsInstance, DomainMember may be used by other Murano Applications). For me this looks much, much better than the current implicit state machine which we run based on XML rules. What do you think about this approach, folks? Do you think it will be easily understood by application developers? Will it be easy to write workflows this way
Re: [openstack-dev] [Murano] Object-oriented approach for defining Murano Applications
Hi Stan, It is good that we are on a common ground here :) Of course this can be done by Heat. In fact - it will be, in the very same manner as it always was, I am pretty sure we've discussed this many times already. When Heat Software config is fully implemented, it will be possible to use it instead of our Agent execution plans for software configuration - it the very same manner as we use regular heat templates for resource allocation. Heat does indeed support template composition - but we don't want our end-users to do learn how to do that: we want them just to combine existing application on higher-level. Murano will use the template composition under the hood, but only in the way which is designed by application publisher. If the publisher has decided to configure the software with using Heat Software Config, then this option will be used. If some other (probably some legacy ) way of doing this was preferred, Murano should be able to support that and allow to create such workflows. Also, there may be workflow steps which are not covered by Heat by design. For example, application publisher may include creating instance snapshots, data migrations, backups etc into the deployment or maintenance workflows. I don't see how these may be done by Heat, while Murano should definitely support this scenarios. So, as a conclusion, Murano should not be though of as a Heat alternative: it is a different tool located on the different layer of the stack, aiming different user audience - and, the most important - using the Heat underneath. -- Regards, Alexander Tivelkov On Mon, Feb 24, 2014 at 8:36 PM, Stan Lagun sla...@mirantis.com wrote: Hi Alex, Personally I like the approach and how you explain it. I just would like to know your opinion on how this is better from someone write Heat template that creates Active Directory lets say with one primary and one secondary controller and then publish it somewhere. Since Heat do supports software configuration as of late and has concept of environments [1] that Steven Hardy generously pointed out in another mailing thread that can be used for composition as well it seems like everything you said can be done by Heat alone [1]: http://hardysteven.blogspot.co.uk/2013/10/heat-providersenvironments-101-ive.html On Mon, Feb 24, 2014 at 7:51 PM, Alexander Tivelkov ativel...@mirantis.com wrote: Sorry folks, I didn't put the proper image url. Here it is: https://creately.com/diagram/hrxk86gv2/kvbckU5hne8C0r0sofJDdtYgxc%3D -- Regards, Alexander Tivelkov On Mon, Feb 24, 2014 at 7:39 PM, Alexander Tivelkov ativel...@mirantis.com wrote: Hi, I would like to initiate one more discussion about an approach we selected to solve a particular problem in Murano. The problem statement is the following: We have multiple entities like low level resources and high level application definitions. Each entity does some specific actions for example to create a VM or deploy application configuration. We want each entity's workflow code reusable in order to simplify development for a new application as the current approach with XML based rules requires significant efforts. After internal discussions inside Murano team we come up to the solution which uses a well known programmatic concept - classes, their inheritance and composition. In this thread I would like to share our ideas and discuss the implementation details. We want to represent each and every entity being manipulated by Murano, as an instance of some class. These classes will define structure of the entities and their behavior. Different entities may be combined together, interacting with each other, forming a composite environment. The inheritance may be used to extract common structure and functionality into generic superclasses, while having their subclasses to define only their specific attributes and actions. This approach is better to explain on some example. Let's consider the Active Directory windows service. This is one of the currently present Murano Applications, and its structure and deployment workflow is pretty complex. Let's see how it may be simplified by using the proposed object-oriented approach. First, let's just describe an Active Directory service in plain English. Active Directory service consists of several Controllers: exactly one Primary Domain Controller and, optionally, several Secondary Domain Controllers. Controllers (both primary and Secondary) are special Windows Instances, having an active directory server role activated. Their specific difference is in the configuration scripts which are executed on them after the roles are activated. Also, Secondary Domain Controllers have an ability to join to a domain, while Primary Domain Controller cannot do it. Windows Instances are regular machines having some limitations on the their images (it should, obviously, be Windows image) and hardware flavor (windows is usually
Re: [openstack-dev] [OpenStack-dev][HEAT][Windows] Does HEAT support provisioning windows cluster
Hi Jay, Windows support in Heat is being developed, but is not complete yet, afaik. You may already use Cloudbase Init to do the post-deploy actions on windows - check [1] for the details. Meanwhile, running a windows cluster is a much more complicated task then just deploying a number of windows instances (if I understand you correctly and you speak about Microsoft Failover Cluster, see [2]): to build it in the cloud you will have to execute quite a complex workflow after the nodes are actually deployed, which is not possible with Heat (at least for now). Murano project ([3]) does this on top of Heat, as it was initially designed as Windows Data Center as a Service, so I suggest you too take a look at it. You may also check this video ([4]) which demonstrates how Murano is used to deploy a failover cluster of Windows 2012 with a clustered MS SQL server on top of it. [1] http://wiki.cloudbase.it/heat-windows [2] http://technet.microsoft.com/library/hh831579 [3] https://wiki.openstack.org/Murano [4] http://www.youtube.com/watch?v=Y_CmrZfKy18 -- Regards, Alexander Tivelkov On Thu, Feb 20, 2014 at 2:02 PM, Jay Lau jay.lau@gmail.com wrote: Hi, Does HEAT support provisioning windows cluster? If so, can I also use user-data to do some post install work for windows instance? Is there any example template for this? Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Repositoris re-organization
Hi Ruslan, Thanks for your feedback. I completely agree with these arguments: actually, these were the reasons why I've initiated this discussion. Team, let's discuss this on the IRC meeting today. -- Regards, Alexander Tivelkov On Tue, Feb 18, 2014 at 5:55 PM, Ruslan Kamaldinov rkamaldi...@mirantis.com wrote: I'd suggest to reduce number of Murano repositories for several reasons: * All other OpenStack projects have a single repo per project. While this point might look like something not worth mentioning, it's really important: - unified project structure simplifies life for new developers. once they get familiar with one project, they can expect something similar from another project - unified project structure simplifies life for deployers. similar project structure simplifies packaging and deployment automation * Another important reason is to simplify gated testing. Just take a look at Solum layout [1], they have everything needed (contrib, functionaltests) to run dvsm job in a single repo. One simple job definition [2] allows to install Solum in DevStack and run Tempest tests for Solum. * As a side-effect, this approach will improve integrity of project components. Having murano-common in the same repo with other components will help to catch integration issues earlier. In an ideal world there will be only the following repos: - murano (api, common, conductor, docs, repository, tests) - python-muranoclient - murano-dashboard - murano-agent - puppet-murano (optional, but nice to have) [1] https://github.com/stackforge/solum [2] https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/jenkins_job_builder/config/solum.yaml Thanks, Ruslan On Thu, Feb 6, 2014 at 3:14 PM, Serg Melikyan smelik...@mirantis.comwrote: Hi, Alexander, In general I am completely agree with Clint and Robert, and as one of contributors of Murano I don't see any practical reasons for repositories reorganization. And regarding of your proposal I have a few thoughts that I would like to share below: This enourmous amount of repositories adds too much infrustructural complexity Creating a new repository is a quick, easy and completely automated procedure that requires only simple commit to Zuul configuration. All infrastructure related to repositories is handled by Openstack CI and supported by Openstack Infra Team, and actually don't require anything from project development team. About what infrastructure complexity you are talking about? I actually think keeping them separate is a great way to make sure you have ongoing API stability. (c) Clint I would like to share a little statistic gathered by Stan Lagun a little time ago regarding repositories count in different PaaS solution. If you are concerned about large number of repositories used by Murano, you will be quite amused: - https://github.com/heroku - 275 - https://github.com/cloudfoundry - 132 - https://github.com/openshift - 49 - https://github.com/CloudifySource - 46 First of all, I would suggest to have a single reposository for all the three main components of Murano: main murano API (the contents of the present), workflow execution engine (currently murano-conductor; also it was suggested to rename the component itself to murano-engine for more consistent naming) and metadata repository (currently murano-repository). *murano-api* and *murano-repository* have many things in common, they are both present HTTP API to the user, and I hope would be rewritten to common framework (Pecan?). But *murano-conductor* have only one thing in common with other two components: code shared under *murano-common*. That repository may be eventually eliminated by moving to Oslo (as it should be done). Also, it has been suggested to move our agents (both windows and unified python) into the main repository as well - just to put them into a separate subfolder. I don't see any reasons why they should be separated from core Murano: I don't believe we are going to have any third-party implementations of our Unified agent proposals, while this possibility was the main reason for separatinng them. Main reason for murano-agent to have separate repository was not a possibility to have another implementation, but that all sources that should be able to be built as package, have tests and can be uploaded to PyPI (or any other gate job) should be placed in different repository. OpenStack CI have several rules regarding how repositories should be organized to support running different gate jobs. For example, to run tests *tox.ini* is need to be present in root directory, to build package *setup.py* should be present in root directory. So we could not simply move them to separate directories in main repository and have same capabilities as in separate repository. Next, deployment scripts and auto-generated docs: are there reasons why they should
Re: [openstack-dev] [Murano] Need a new DSL for Murano
Hi Joshua, Thank you very much for you feedback! This is really a great question, and it was the first question we've asked ourselves when we started thinking about this new design. We've considered both options: to have our own syntax (python-like, java-like, something-else-like) or to use YAML. We've chosen the latest, and here are the reasons. The most important moment: this is not a general-purpose language, but a domain-specific one. And Murano's domain is manipulating with complex nested data-structures, which are usually represented by YAML. Murano's environments are in YAML. Murano's VM-side commands are wrapped into YAML. The primary building blocks of Murano - Heat templates - are written on YAML. Actually, at a simplified level murano's workflows can be thought of as algorithms that just generate yaml fragments. So, it would be beneficial for Murano to manipulate with YAML-constructs at the DSL level. If we use YAML notation, yaml-blocks become first-class citizens in the language, while in a regular python-like language they would be just formatted-strings. For example, look at this code snippet which generates a heat template to deploy an instance: http://paste.openstack.org/show/65725/ As you may see, the code on lines 7-11 is a Heat-template, seamlessly embedded inside Murano's workflow code. It has the variables right inside the template, and the Murano engine will substitute them with a user-specified (or workflow-computed) data Another reason for YAML: the notation is very easy to extend: you'll just have to add some new predefined key and a handler for its value: the format will not be broken, so older code will run out of the box, and even the newer code will most probably run fine on the older parser (the unknown sections will simply be ignored). This will allow us to extend the language without making any breaking-changes. The regular languages do not provide this flexibility: they will have problems if detecting unrecognised lexems or constructs. Last but not least: the perception. You are absolutely right when you say that this is actually a full programming language. However, we don't want to rush all its capabilities to unprepared developers. If some developer does not want any complexity, they may think about it as about some fancy configuration markup language: a declarative, heat-template-like header followed by a sequence of simple actions. And only if needed the power comes at your service: variable assignments, flow control, flexible data contracts, complex compositions, inheritance trees.. I can imagine a lot of scary stuff here J But at the same time, YAML's indent-based syntax will look familiar to python developers. Yes, everything comes at cost, and yaml may seem a bit bulky at the first glance. But I believe that people will get used to it soon enough, and the benefits are really important. I hope this answers your concern. We'll come up with more examples and ideas: this thing has just emerged, nothing is set in stone yet, I am actively seeking for feedback and ideas. So thanks a loot for your question, I really appreciate it. -- Regards, Alexander Tivelkov On Fri, Feb 14, 2014 at 6:41 PM, Joshua Harlow harlo...@yahoo-inc.comwrote: An honest question, U are mentioning what appears to be the basis for a full programming language (variables, calling other workflows - similar to functions) but then u mention this is being stuffed into yaml. Why? It appears like u might as well spend the effort and define a grammar and simplistic language that is not stuffed inside yaml. Shoving one into yaml syntax seems like it gets u none of the benefits of syntax checking, parsing, validation (highlighting...) and all the pain of yaml. Something doesn't seem right about the approach of creating languages inside the yaml format (in a way it becomes like xsl, yet xsl at least has a spec and is well defined). My 2 cents Sent from my really tiny device... On Feb 14, 2014, at 7:22 PM, Alexander Tivelkov ativel...@mirantis.com wrote: Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance. Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They are caused by the fact that it was never intended for use for Application Catalog use-cases. I'll briefly touch these drawbacks first: 1. Murano's workflow engine is actually a state machine, however the workflow markup does not explicitly define the states and transitions. 2. There is no data isolation within any environment, which causes both potential security vulnerabilities and unpredictable workflow behaviours. 3. There is no easy way to reuse
[openstack-dev] [Murano] Need a new DSL for Murano
Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance. Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They are caused by the fact that it was never intended for use for Application Catalog use-cases. I'll briefly touch these drawbacks first: 1. Murano's workflow engine is actually a state machine, however the workflow markup does not explicitly define the states and transitions. 2. There is no data isolation within any environment, which causes both potential security vulnerabilities and unpredictable workflow behaviours. 3. There is no easy way to reuse the workflows and their related procedures between several applications. 4. The markup uses JSONPath, which relies on Python's 'eval' function. This is insecure and has to be avoided. 5. 5. The workflow markup is XML-based, which is not a common practice in the OpenStack community. So, it turns out that we have to design and implement a new workflow definition notation, which will not have any of the issues mentioned above. At the same time, it should still allow to fully specify the configuration of any third-party Application, its dependencies with other Applications and define specific actions which are required for Application deployment, configuration and life cycle management. This new notation should allow to do the following: - List all the required configuration parameters and dependencies for a given application - Validate user input and match it to the defined parameters - Define specific deployment actions and their execution order - Define behaviors to handle the events of changes in application's environment Also, it should satisfy the following requirements: - Minimize the amount of configuration for common application parts, i.e. reuse existing configuration parts and add only difference specific to the application. - Allow to use different deployment tools with using the same markup constructs. i.e. provide a high-level abstraction on the underlying tools (heat, shell, chef, puppet etc) - For security reasons it should NOT allow to execute arbitrary operations - i.e. should allow to run only predefined set of meaningful configuration actions. So, I would suggest to introduce a simple and domain specific notation which would satisfy these needs: - Application dependencies and configuration properties are defined declaratively, in a way similar to how it is done in Heat templates. - Each property has special constraints and rules, allowing to validate the input and applications relationship within the environment. - The workflows are defined in imperative way: as a sequence of actions or method calls. This may include assigning data variables or calling the workflows of other applications. - All of these may be packaged in a YAML format. The example may look something like this [1] The final version may become a bit more complicated, but as the starting point this should look fine. I suggest to cover this in more details on our next IRC meeting on Tuesday. Any feedback or suggestions are appreciated. [1] https://etherpad.openstack.org/p/murano-new-dsl-example -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Community meeting agenda - 02/11/2014
Hi, This is just a reminder that we are going to have a weekly meeting of Murano team in IRC (#openstack-meeting-alt) today at 17:00 UTC (9am PST) . The agenda can be found here: https://wiki.openstack.org/wiki/Meetings/MuranoAgenda#Agenda Feel free to add anything you want to discuss. -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Can we migrate to oslo.messaging?
Hi Joshua, Currently we are not 3.x compatible, at least not all of the modules. Migration to python3 should eventually be done, but for now it is the least of our concerns, while relying on a non-standard messaging solution instead of community-approved one is likely to cause problems, both in terms of code stability and formal incubation-related requirements. -- Regards, Alexander Tivelkov On Tue, Feb 11, 2014 at 6:52 PM, Joshua Harlow harlo...@yahoo-inc.comwrote: Is murano python3.x compatible, from what I understand oslo.messaging isn't (yet). If murano is supporting python3.x then brining in oslo.messaging might make it hard for murano to be 3.x compatible. Maybe not a problem (I'm not sure of muranos python version support). From: Serg Melikyan smelik...@mirantis.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: Tuesday, February 11, 2014 at 5:05 AM To: OpenStack Development Mailing List OpenStack-dev@lists.openstack.org Subject: [openstack-dev] [Murano] Can we migrate to oslo.messaging? oslo.messaging http://github.com/openstack/oslo.messaging is a library that provides RPC and Notifications API, they are part of the same library for mostly historical reasons. One of the major goals of *oslo.messaging* is to provide clean RPC and Notification API without any trace of messaging queue concepts (but two of most advanced drivers used by oslo.messaging is actually based on AMQP: RabbitMQ and QPID). We were designing Murano on messaging queue concepts using some AMQP/RabbitMQ specific features, like queue TTL. Since we never considered communications between our components in terms of RPC or Notifications and always thought about them as message exchange through broker it has influenced our components architecture. In Murano we use simple wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messaging around Puka https://github.com/majek/puka (RabbitMQ client with most simple and thoughtful async model) that is used in all our components. We forked Puka https://github.com/istalker2/puka since we had specific requirements to SSL and could not yet merge our workhttps://github.com/majek/puka/pull/43 back to master. Can we abandon our own wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messagingaround our own fork of Puka https://github.com/istalker2/puka in favor of *oslo.messaging*? * Yes*, but this migration may be tricky. I believe we can migrate to *oslo.messaging* in a week or so*.* I had played with *oslo.messaging* emulating our current communication patterns with *oslo.messaging*, and I am certain that current implementation can be migrated to *oslo.messaging**. *But I am not sure that *oslo.messaging* may be easily suited to all future use-cases that we plan to cover in a few next releases without major contributions. Please, try to respond with any questions related to *oslo.messaging* implementation and how it can be fitted with certain use-case. Below, I tried to describe our current use-cases and what specific MQ features we are using, how they may be implemented with *oslo.messaging *and with what limitations we will face. Use-Case Murano has several components with communications between them based on messaging queue: *murano-api* - *murano-conductor:* 1. *murano-api* sends deployment tasks to murano-conductor *murano-conductor* - *murano-api:* 1. *murano-conductor* reports to *murano-api* task progress during processing 2. after processing, *murano-conductor* sends results to *murano-api* *murano-conductor *-* murano-agent:* 1. during task processing *murano-conductor* sends execution plans with commands to *murano-agent.* Note: each of mentioned components above may have more than one instance. One of great messaging queue specific that we heavily use is a idea of queue itself, messages sent to component will be handled any time soon as at least one instance would be started. For example, in case of *murano-agent*, message is sent even before *murano-agent* is started. Another one is queue life-time, we control life-time of *murano-agent*queues to exclude overflow of MQ server with queues that is not used anymore. One thing is also worse to mention: *murano-conductor* communicates with several components at the same time: process several tasks at the same time, during task processing *murano-conductor* sends progress notifications to *murano-api* and execution plans to *murano-agent*. Implementation Please, refer to Conceptshttps://wiki.openstack.org/wiki/Oslo/Messaging#Concepts section of *oslo.messaging* Wiki before further reading to grasp key concepts expressed in *oslo.messaging* library. In short, using RPC API we can 'call' server synchronously and receive some result, or 'cast' asynchronously (no result is returned). Using
[openstack-dev] [Murano] Murano meeting minutes - Feb 4
Hi, Thanks everyone who have joined Murano weekly meeting. These are the meeting minutes and logs: http://eavesdrop.openstack.org/meetings/murano/2014/murano.2014-02-04-17.03.txt http://eavesdrop.openstack.org/meetings/murano/2014/murano.2014-01-14-17.03.log.html As we were running out of time while the repository reorganisation discussion was not completed, we have agreed that we will continue the discussion in the mailing list. Please, feel free to express your opinions on the topic. Thanks! -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Community meeting agenda - 02/04/2014
Hi, This is just a reminder that we are going to have a weekly meeting of Murano team in IRC (#openstack-meeting-alt) on Feb, 4 at 17:00 UTC (9am PST) . The agenda can be found here: https://wiki.openstack.org/wiki/Meetings/MuranoAgenda#Agenda Feel free to add anything you want to discuss. -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Problem with Allow Address Pairs in Murano Workflows
Hi folks, It seems like Murano networking workflow has a problem with the way how it uses the allowed_address_pairs property of Neutron ports. This problem causes an intermittent bug preventing Microsoft SQL Server Cluster application to be deployed in Murano. We have a bug reported about that - [1]. Seems like this issue became a problem since the time when we introduced the advanced networking, i.e. the algorithm which attempts to automatically pick the proper networking setting for the newly created environment. Deployment workflow of MS SQL Server Cluster (and, wider, any Murano Application relying on the virtual ip address) uses Neutron's Allowed Address Pairs feature ([2]) to specify its virtual ip address, so Neutron allows the calls to this address through the ports of application's machines. However, there is a limitation: Neutron does not allow to specify the address to be equal to the fixed ip address of the port (see the first note at [3]). Murano does not assign the ip addresses of any ports explicitly and relies on the automatic ip allocation provided by Neutron. In the situation where the fixed IP address is not defined, but allowed_address_pair is set, I would expect Neutron do a little analysis and not to allocate the address provided in the allowed pair as the fixed IP. But Neutron does not do it - and the exception is thrown. However, even the worse situation may happen if such an address conflict appears not on a single port, but on the two different ones: when the ip of allow_address_pair of port A is equal to the static ip of port B. This situation is perfectly fine from Neutron's point of view, and no exception is thrown. However, later, during the configuration of the cluster, the virtual IP will point to one of the real ports - which may cause two machine sharing the same actual ip at the same time. You may guess the consequences. So, we need to find some working solution for this situation. The obvious one would be to exclude the virtual ip address from the allocation pools of the subnet, being generated for the environment. This may be tricky (as the cidrs for subnets are picked automatically), but still definitely doable. The problem which I see here is that we will have to do it at the time when the environment is created - i.e. run Neutron API calls from the Dashboard (but we have to do it anyway now, as we check the user's input for cluster ip to match the target subnet). Another solution is to remove the option for user to specify the virtual ip at all, and allocate this IP at the runtime of the workflow, when all the ports of the cluster's instances are already created and their IP known. I don't know if there is a real use-case requiring the user to know the virtual ip in advance and be able to control its value. If there is no such scenario, then we may just hide it, and it will simplify things a lot. May be something else is possible as well. Any ideas are welcome [1] https://bugs.launchpad.net/murano/+bug/1274636 [2] https://blueprints.launchpad.net/neutron/+spec/allowed-address-pairs [3] http://docs.openstack.org/admin-guide-cloud/content/section_allowed_address_pairs_workflow.html -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Proposed Logging Standards
Very interested, thanks a lot for this topic. Will work on bringing all of this to Murano -- Regards, Alexander Tivelkov On Tue, Jan 28, 2014 at 6:45 AM, Sergey Lukjanov slukja...@mirantis.comwrote: FYI it was added to the project meeting agenda - https://wiki.openstack.org/wiki/Meetings/ProjectMeeting On Tue, Jan 28, 2014 at 3:42 PM, Sergey Lukjanov slukja...@mirantis.comwrote: Hi Sean, it's great that you're catching this up. I'd like to participate. I don't know how much time I'll be able to dedicate on it, but at least I'm ready for reviews and pushing it to Savanna. Thanks! On Tue, Jan 28, 2014 at 3:21 PM, Sean Dague s...@dague.net wrote: On 01/27/2014 09:57 PM, Christopher Yeoh wrote: On Tue, Jan 28, 2014 at 12:55 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote: Hi Sean, I'm currently working on moving away from the built-in logging to use log_config=filename and the python logging framework so that we can start shipping to logstash/sentry/insert other useful tool here. I'd be very interested in getting involved in this, especially from a why do we have log messages that are split across multiple lines perspective! Do we have many that aren't either DEBUG or TRACE? I thought we were pretty clean there. Cheers, Matt P.S. FWIW, I'd also welcome details on what the Audit level gives us that the others don't... :) Well as far as I can tell the AUDIT level was a prior drive by contribution that's not being actively maintained. Honestly, I think we should probably rip it out, because I don't see any in tree tooling to use it, and it's horribly inconsistent. For the uses I've seen of it in the nova api code INFO would be perfectly fine in place of AUDIT. I'd be happy to help out with patches to cleanup the logging in n-api. One other thing to look at - I've noticed with logs is that when something like glanceclient code (just as an example) is called from nova, we can get ERROR level messages for say image not found when its actually perfectly expected that this will occur. I'm not sure if we should be changing the error level in glanceclient or just forcing any error logging in glanceclient when called from Nova to a lower level though. It's now changed in glanceclient - https://review.openstack.org/#/c/67744/ - it should be gone in the gate logs, and will be gone for everyone once a new release is out. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc. -- Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Repositoris re-organization
Clint, Rob, Thanks a lot for your input: that's really a good point, and we didn't consider it before, while we definitely should. Team, Let's discuss this topic again before making any final decisions. -- Regards, Alexander Tivelkov 2014/1/24 Robert Collins robe...@robertcollins.net On 24 January 2014 22:26, Clint Byrum cl...@fewbar.com wrote: This enourmous amount of repositories adds too much infrustructural complexity, and maintaining the changes in in consistent and reliable manner becomes a really tricky tasks. We often have changes which require modifing two or more repositories - and thus we have to make several changesets in gerrit, targeting different repositories. Quite often the As does adding any feature with e.g. networking - change neutron, neutronclient and nova, or block storage, change cinder, cinderclient and nova... This isn't complexity - it's not the connecting together of different things in inappropriate ways - its really purity, you're having to treat each thing as a stable library API. dependencies between these changesets are not obvious, the patches get reviewed and approved on wrong order (yes, this also questions the quality of the code review, but that is a different topic), which causes in inconsostent state of the repositories. Actually it says your tests are insufficient, otherwise things wouldn't be able to land :). So, as somebody who does not run Murano, but who does care a lot about continuous delivery, I actually think keeping them separate is a great way to make sure you have ongoing API stability. +1 bet me to that by just minutes:) Since all of those pieces can run on different machines, having the APIs able to handle both the old way and the new way is quite helpful in a large scale roll out where you want to keep things running while you update. Anyway, that may not matter much, but it is one way to think about it. Indeed :) -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Repositoris re-organization
additional which will remain on stackforge), with clear separation of concerns. There may be technical issues in doing this mergement (we do not want to loose revision history, do we?), but they should be solvable (I'll write to infra asking on what is possible and what is not), but in general this is the direction in which we should be moving, as it seems to me. -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] Windows Support
Hi! These are great news indeed: I am glad to hear that Windows support comes to Heat at last! Meanwhile, you may always take a look at what was done in this area in Murano: it has started as Windows-deployment tool based on top of Heat, so we have covered lots of windows-related issues and have tons of expertise in the subject. This expertise may be not 100%-relevant for pure Heat usage (as Murano uses its own workflow DSL and vm-side agent), however some of our steps can be definitely repeated. Feel free to check Murano's docs ( https://wiki.openstack.org/wiki/Murano) and ask any questions in mailing list or IRC (#murano) -- Regards, Alexander Tivelkov 2014/1/9 Steven Hardy sha...@redhat.com Hi Winson, On Wed, Jan 08, 2014 at 08:41:16PM +, Chan, Winson C wrote: Does anybody know if this blueprint is being actively work on? https://blueprints.launchpad.net/heat/+spec/windows-instances If this is not active, can I take ownership of this blueprint? My team wants to add support for Windows in Heat for our internal deployment. Ha, that BP has been unassigned for nearly a year, then two folks want to take it on the same day, what are the chances! :) Alex Pilotti pinged me on IRC yesterday asking about it, and offered to take ownership of the BP, so I assigned it to him. That said, I'm pretty sure there is scope for breaking down the work so you can take on some tasks - we just need to evaluate what needs to be done and raise some child blueprints so the effort can be distributed. The steps I can think of, unless they have already been done by folks: - Evaluate bootstrap agent (I'd assumed cloudbase-init would work, which Alex indicated was the case yesterday) with Heat generated userdata. - Figure out if we have path issues in userdata/part-handler which need resolving - Work out what we do with heat-cfntools: - Add support for windows? - Figure out a way to work with a fork of cfnbootstrap (which already works on windows I think (ref https://bugs.launchpad.net/heat/+bug/1103811) - Support some other method for secondary post-boot customization (e.g just use cloudbase-init, or integrate with some other existing agent) - Document preparation of a Heat-enabled Windows image - Windows example templates and user documentation There's probably more stuff I haven't considered - hopefully you can connect with Alex, work out a way to divide the effort and raise some new BPs? To me the biggest unknown is the in-instance agent thing, but tbh I've not really looked at it in much detail so I'd be happy to hear peoples thoughts and experiences. Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Team meeting reminder - December 3
Hi! This is just a reminder about the regular meeting of Murano-team in IRC. The meeting will be held in #openstack-meeting-alt channel at 10am Pacific. The complete agenda of the meeting is available here: https://wiki.openstack.org/wiki/Meetings/MuranoAgenda -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Murano Weekly Meeting
Hi, This is just a reminder that we are NOT having the IRC meeting today due to reschedule: the meeting is moved to Tuesday 17:00 UTC. So, see you in IRC tomorrow! -- Regards, Alexander Tivelkov 2013/11/13 Serg Melikyan smelik...@mirantis.com Hello, OpenStack developers! We rescheduled our Murano IRC meeting to Tuesday, 17:00 UTC due to Daylight Saving Time. -- Serg Melikyan, Senior Software Engineer at Mirantis, Inc. http://mirantis.com | smelik...@mirantis.com +7 (495) 640-4904, 0261 +7 (903) 156-0836 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Implementing Elastic Applications
Hi, I believe that we restricted to have a single solution only: Murano is an Application Catalog now, and Catalog is the thing where multiple similar solutions can be present, and the user makes the final decision on what to pick for their environments. So, I would suggest to bundle a solution based on OS::Neutron::LoadBalancer - and have a blueprint describing that homemade HAProxy-based solution - someone will implement it sooner or later, as the demand for such a service clearly exists (and will definitely increase when we introduce the ability to share a single VM for multiple applications). -- Regards, Alexander Tivelkov 2013/11/15 Serg Melikyan smelik...@mirantis.com Murano has several applications which support scaling via load-balancing, this applications (Internet Information Services Web Farm, ASP.NETApplication Web Farm) currently are based on Heat http://launchpad.net/heat, particularly on resource called AWS::ElasticLoadBalancing::LoadBalancerhttp://docs.openstack.org/developer/heat/template_guide/cfn.html#AWS::ElasticLoadBalancing::LoadBalancer, that currently does not supporthttp://docs.openstack.org/developer/heat/template_guide/cfn.html#AWS::ElasticLoadBalancing::LoadBalancer-props specification of any network related parameters. Inability to specify network related params leads to incorrect behavior during deployment in tenants with advanced Quantum deployment configuration, like Per-tenant Routers with Private Networks and this makes deployment of our ** Web Farm* applications to fail. We need to resolve issues with our ** Web Farm*, and make this applications to be reference implementation for elastic applications in Murano. This issue may be resolved in three ways: via extending configuration capabilities of AWS::ElasticLoadBalancing::LoadBalancerhttp://docs.openstack.org/developer/heat/template_guide/cfn.html#AWS::ElasticLoadBalancing::LoadBalancer, using another implementation of load balancing in Heat - OS::Neutron::LoadBalancerhttp://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Neutron::LoadBalancer or via implementing own load balancing application (that going to balance other apllications), for example based on HAProxy http://haproxy.1wt.eu/ (as all previous ones). Please, respond with your thoughts on the question: *Which implementation we should use to resolve issue with our Web Farm applications and why?*. Below you can find more details about each of the options. *AWS::ElasticLoadBalancing::LoadBalancer* AWS::ElasticLoadBalancing::LoadBalancer is Amazon Cloud Formation compatible resource that implements load balancer via hard-coded nested stackhttps://github.com/openstack/heat/blob/master/heat/engine/resources/loadbalancer.py#L24that deploys and configures HAProxy. This resource requires specific image with CFN Tools https://github.com/openstack/heat-cfntools and specific name *F17-x86_64-cfntools* available in Glance. It's look like we miss implementation of only one property in this resource - Subnets. *OS::Neutron::LoadBalancer* OS::Neutron::LoadBalancerhttp://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Neutron::LoadBalanceris another Heat resource that implements load balancer. This resource is based on Load Balancer as a Service feature in Neutronhttps://wiki.openstack.org/wiki/Neutron/LBaaS. OS::Neutron::LoadBalancer is much more configurable and sophisticated but underlying implementation makes usage of this resource quite complex. LBaaS is a set of services installed and configured as a part of Neutron. Fuel does not support LBaaS; Devstack has support for LBaaS, but LBaaS not installed by default with Neutron. *Own, Based on HAProxy* We may implement load-balancer as a regular application in Murano using HAProxy http://haproxy.1wt.eu/. This service may look like our Active Directory application with almost same user-expirience. User may create load-balancer inside of the environment and join any web-application (with any number of instances) directly to load-balancer. Load-balancer may be also implemented on Conductor workflows level, this implementation strategy not going to change user-experience (in fact we changing only underlying implementation details for our * Web Farm applications, without introducing new ones). -- Serg Melikyan, Senior Software Engineer at Mirantis, Inc. http://mirantis.com | smelik...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Designing and implementing Murano (App Catalog for OpenStack)
Hey Stackers, During HK summit we’ve been discussing an approach to implement app catalog for Openstack. We’ve seen certain interest in the topic and heard from few people that they are working on similar concepts outside of community. Apparently Openstack is now mature enough to have a need for an application catalog which will fulfil several needs of cloud admins and users on platform level: - an integration point for applications and services running on top of OS - the level of management of lifecycle of applications, distribution channels and billing - self-service provisioning for applications Few weeks ago we proposed to extend the mission of Murano to be an Application Catalog for Openstack, simultaneously we decided to contribute several parts of the project to other OpenStack projects, mainly Heat and Mistral. Our intention is to make the mission and scope of the project as lean as possible, at the same time leveraging as much of the existing Openstack services as possible. As many others we’re excited about Solum project and looking forward to integrate with Solum and contribute there. Now the most exciting part - we are looking for contributors! Use cases, architecture, blueprints, code - anything would be helpful. Right now is the best time to start, as we are just started figuring out definition of Application Catalog, getting the requirements, considering use cases and design concepts. The project from day one is run on OpenStack infrastructure, following all standard OpenStack processes and practices and we’re really making an effort to make it very open. We believe that this is an initiative which can bring a lot of value to the community, let’s try to obtain synergy working on this all together. The earlier you express your interest explicitly, the more impact you’re going to have on the roadmap. The high-level overview of the proposal can be found at our wiki: https://wiki.openstack.org/wiki/Murano/ApplicationCatalog This is a work-in-progress document, and it is going to be changed - so your feedback is very welcome right now. We’ve created several etherpads to collaborate and gather the requirements together. Feel free to add anything which seems important for you: https://etherpad.openstack.org/p/AppCatalogUI - requirements for UI https://etherpad.openstack.org/p/AppCatalogRoadmap - roadmap with features and milestones https://etherpad.openstack.org/p/AppCatalogUseCases - some use-cases, giving examples of how different users interact with the Application Catalog Feel free to add anything to these etherpads - and share any ideas you might have with all the community. Looking forward for your participation! Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum/Heat] Is Solum really necessary?
Hi, That's an important question, and I've seen it being asked many times before, often regarding the Murano project, which also hides Heat templates under the hood: people were asking why do they need yet another abstraction layer on top of the familiar and powerful tool such as Heat. I believe that the difference is the target audience of the projects. It seems to me that Heat's primary users are the people who will write their own templates - or use the existing ones but having a deep understanding of how their work. Meanwhile, the end-users of Solum are application developers, they do not need (and, probably, do not want at all) to worry about infrastructure-specific tools, frameworks and APIs - and they are probably not going to write the Heat (or HOT) templates on their own: they need a higher-level tooling for that. And that is exactly the place where Solum will come into play, generating these templates for them. -- Regards, Alexander Tivelkov 2013/11/14 Georgy Okrokvertskhov gokrokvertsk...@mirantis.com Hi, I think that Heat is mostly focused on deployment even with new software configs and convergence. HOT template is quite static description of desired state we want to achieve and it is up to Heat engine how to achieve this state. Solum is focused on managing the process of converting source code to some deployable entity (image or container). The power of Solum is an ability to fully describe and control the process of building and testing of application. Some of the stages of build and testing process might require actual deployment and stack creation, but this is not an ultimate goal of the Solum. If someone will try to use just Heat for building process description they will figure out quickly that they need different templates for different build\testing stages. As Heat itself can't modify templates you will need some external mechanism for template creation, and this is what Solum actually does. Thanks Georgy On Thu, Nov 14, 2013 at 11:08 AM, Christopher Armstrong chris.armstr...@rackspace.com wrote: On Thu, Nov 14, 2013 at 11:04 AM, Sam Alba sam.a...@gmail.com wrote: Hi Jay, I think Heat is an ingredient for Solum. When you build a PaaS, you need to control the app at different levels: #1 describing your app (basically your stack) #2 Pushing your code #3 Deploying it #4 Controlling the runtime (restart, get logs, scale, changing resources allocation, etc...) I think Heat is a major component for step 3. But I think Heat's job ends at the end of the deployment (the status of the stack is COMPLETED in Heat after processing the template correctly). It's nice though to rely on Heat's template generation for describing the stack, it's one more thing to delegate to Heat. In other words, I see Heat as an engine for deployment (at least in the context of Solum) and have something on top to manage the other steps. I'd say that Heat does (or should do) more than just the initial deployment -- especially with recent discussion around healing / convergence. -- IRC: radix Christopher Armstrong Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Georgy Okrokvertskhov Technical Program Manager, Cloud and Infrastructure Services, Mirantis http://www.mirantis.com Tel. +1 650 963 9828 Mob. +1 650 996 3284 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Summit activities
Hi folks, OpenStack summit has begun, and we have pretty large part of the Murano team here in Hong Kong. You may always find me or anybody else from Mirantis - and we will be happy to answer any questions. Also, we have a Lighting Talk planned for tomorrow at 1:35 pm - I will quickly cover the updated vision of our mission. Also, on Friday at 11 am, as part of unconference track we'll have a session dedicated to discussing of Murano Application catalog use cases and roadmap. Also, feel free to come to Mirantis booth - we have the demo of our products there, and will be happy to answer any questions as well. See you at the summit! -- Kind Regards, Alexander Tivelkov Principal Software Engineer OpenStack Platform Product division Mirantis, Inc +7(495) 640 4904, ext 0236 +7-926-267-37-97(cell) Vorontsovskaya street 35 B, building 3, Moscow, Russia. ativel...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Trove][Savanna][Murano] Unified Agent proposal discussion at Summit
Hi guys, Recently we had several discussions about the guest VM agents: lot's of projects have the similar needs to run some special logic on the side of guest virtual machines. As far as I know, there are such agents in Savanna, Trove, Murano and may be some other projects as well. The obvious idea is to unite the efforts and have the unified solution which may satisfy everybody's needs. We've discussed this topic before with some of the teams, and got the promising-looking idea to create kind of unified agent library and put it in Oslo or some other shared project. We've scheduled an unconference session on the Summit, this Friday at 3:10 pm. Let's continue discussing the idea there: we may gather the common requirements, discuss the basic design concepts etc. See you there! -- Kind Regards, Alexander Tivelkov Principal Software Engineer OpenStack Platform Product division Mirantis, Inc +7(495) 640 4904, ext 0236 +7-926-267-37-97(cell) Vorontsovskaya street 35 B, building 3, Moscow, Russia. ativel...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Team Meeting reminder - October 28
Hi folks This is just a reminder about the regular meeting of Murano-team in IRC. The meeting will be held in #openstack-meeting-alt channel at 8am Pacific. We'll discuss the status of our current delivery (Murano 0.3) and the issues which we recently faced with it. The complete agenda of the meeting is available here: https://wiki.openstack.org/wiki/Meetings/MuranoAgenda -- Kind Regards, Alexander Tivelkov Principal Software Engineer OpenStack Platform Product division Mirantis, Inc +7(495) 640 4904, ext 0236 +7-926-267-37-97(cell) Vorontsovskaya street 35 B, building 3, Moscow, Russia. ativel...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Team Meeting reminder - October 21
Hi folks This is just a reminder about the regular meeting of Murano-team in IRC. The meeting will be held in #openstack-meeting-alt channel at 8am Pacific. We'll discuss our upcoming delivery of Murano 0.3, as well as review some of the proposed changes (mainly the Network configuration) for the next - 0.4 - release. The complete agenda of the meeting is available here: https://wiki.openstack.org/wiki/Meetings/MuranoAgenda -- Kind Regards, Alexander Tivelkov Principal Software Engineer OpenStack Platform Product division Mirantis, Inc +7(495) 640 4904, ext 0236 +7-926-267-37-97(cell) Vorontsovskaya street 35 B, building 3, Moscow, Russia. ativel...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Team Meeting minutes - 10/21
Hi, Thanks everyone who has joined Murano IRC meeting. These are the meeting minutes and the action items: http://eavesdrop.openstack.org/meetings/murano/2013/murano.2013-10-21-15.00.html Complete logs can be found here: http://eavesdrop.openstack.org/meetings/murano/2013/murano.2013-10-21-15.00.log.html -- Kind Regards, Alexander Tivelkov Principal Software Engineer OpenStack Platform Product division Mirantis, Inc +7(495) 640 4904, ext 0236 +7-926-267-37-97(cell) Vorontsovskaya street 35 B, building 3, Moscow, Russia. ativel...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Team meeting reminder - October 14
Hi! This is just a reminder about the regular meeting of Murano-team in IRC. The meeting will be held in #openstack-meeting-alt channel at 8am Pacific. The complete agenda of the meeting is available here: https://wiki.openstack.org/wiki/Meetings/MuranoAgenda -- Kind Regards, Alexander Tivelkov Principal Software Engineer OpenStack Platform Product division Mirantis, Inc +7(495) 640 4904, ext 0236 +7-926-267-37-97(cell) Vorontsovskaya street 35 B, building 3, Moscow, Russia. ativel...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Team meeting reminder - September 30
Hi! This is just a reminder about the regular meeting of Murano-team in IRC. The meeting will be held in #openstack-meeting-alt channel at 8am Pacific. The main topic of the discussion will be Linux Agent and Metadata Repository. The complete agenda is available here: https://wiki.openstack.org/wiki/Meetings/MuranoAgenda -- Kind Regards, Alexander Tivelkov Principal Software Engineer OpenStack Platform Product division Mirantis, Inc +7(495) 640 4904, ext 0236 +7-926-267-37-97(cell) Vorontsovskaya street 35 B, building 3, Moscow, Russia. ativel...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
