Re: [openstack-dev] Time to Samba! :-)
On Sat, Aug 16, 2014 at 11:03 PM, Martinx - ジェームズ wrote: > Hey Stackers, > > I'm wondering here... Samba4 is pretty solid (up coming 4.2 rocks), I'm > using it on a daily basis as an AD DC controller, for both Windows and Linux > Instances! With replication, file system ACLs - cifs, built-in LDAP, dynamic > DNS with Bind9 as a backend (no netbios) and etc... Pretty cool! > > In OpenStack ecosystem, there are awesome solutions like Trove, Solum, > Designate and etc... Amazing times BTW! So, why not try to integrate Samba4, > working as an AD DC, within OpenStack itself?! > > If yes, then, what is the best way/approach to achieve this?! > > I mean, for SQL, we have Trove, for iSCSI, Cinder, Nova uses Libvirt... > Don't you guys think that it is time to have an OpenStack project for LDAP > too? And since Samba4 come with it, plus DNS, AD, Kerberos and etc, I think > that it will be huge if we manage to integrate it with OpenStack. > > I think that it would be nice to have, for example: domains, users and > groups management at Horizon, and each tenant with its own "Administrator" > (not the Keystone "global" admin) (to mange its Samba4 domains), so, they > will be able to fully manage its own account, while allowing Keystone to > authenticate against these users... > > Also, maybe Designate can have support for it too! I don't know for sure... > > Today, I'm doing this "Samba integration" manually, I have an "external" > Samba4, from OpenStack's point of view, then, each tenant/project, have its > own DNS domains, when a instance boots up, I just need to do something like > this (bootstrap): > > -- > echo "127.0.1.1 instance-1.tenant-1.domain-1.com instance-1" >> /etc/hosts > net ads join -U administrator > -- > > To make this work, the instance just needs to use Samba4 AD DC as its Name > Servers, configured at its /etc/resolv.conf, "delivered by DHCP Agent". The > packages `samba-common-bin` and `krb5-user` are also required. Including a > ready to use smb.conf file. > > Then, "ping instance-1.tenant-1.domain-1.com" worldwide! It works for both > IPv4 and IPv6!! > > Also, Samba4 works okay with Disjoint Namespaces, so, each tenant can have > one or more domains and subdomains! Like "*.realm.domain.com, *.domain.com, > *.cloud-net-1.domain.com, *.domain2.com... All dynamic managed by Samba4 and > Bind9! > > What about that?! > > Cheers! > Thiago There are several existing OpenStack projects which can help to leverage Samba support: 1. Manila - it seems to be capable of provisioning and attaching CIFS/SMB shares. I know Samba is more than just a CIFS share, but it is a significant part of it 2. You can use Heat to spin up a VM and configure Samba server 3. You can use Murano to spin up VMs with Samba, LDAP, Kerberos, etc (done with Heat internally) and configure them to work together Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Time to Samba! :-)
On Sun, Aug 17, 2014 at 4:16 AM, Adam Lawson wrote: > Doesn't Murano address this already? Please note that Murano is no longer a windows-as-a-service or smth-as-a-serivce. Murano is an application catalog [1]. But you're absolutely right, this is a perfect use case for Murano - application developer can describe those applications and publish them in catalog, which will enable cloud users to combine those apps together. LDAP, Kerberos, Samba, ActiveDirectory - are applications in terms of Murano. [1] https://wiki.openstack.org/wiki/Murano -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [pbr] help needed fixing bad version= lines in setup.cfg in many projects
On Wed, Aug 20, 2014 at 5:12 AM, Robert Collins wrote: > Hi, in working on pbr I have run into some bad data in our > setup.cfg's. Details are here: > https://etherpad.openstack.org/p/bad-setup-cfg-versions > > Short version: we need to do a missed step in the release project for > a bunch of stable branches, and in some cases master, for a bunch of > projects. > > I'm going to script up an automated push up of reviews for all of > this, so I'm looking for core and stable branch maintainers to help > get the reviews through quickly - as this is a blocker on pbr. > > We don't need to fix branches that won't be sdist'd anymore - so if > its just there for tracking, thats fine. Everything else we should > fix. Robert, Thanks a lot for taking care of stackforge projects too! We (Murano) will apply your changes to our active branches. -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] "heat.conf.sample is not up to date"
On Sun, Aug 24, 2014 at 11:10 AM, Mike Spreitzer wrote: > What I really need to know is what to do when committing a change that > really does require a change in the sample configuration file. Of course I > tried running generate_sample.sh, but `tox -epep8` still complains. What is > the right procedure to get a correct sample committed? BTW, I am doing the > following admittedly risky thing: I run DevStack, and make my changes in > /opt/stack/heat/. Mike, It seems that you have oslo.messaging installed from master (default behavior of Devstack), while heat.sample.config is built for oslo.messaging v 1.3.1. As a quick workaround I'd suggest to downgrade oslo.messaging to 1.3.1 in pep8 virtual environment: $ cd /opt/stack/heat $ source .tox/pep8/bin/activate $ pip install oslo.messaging==1.3.1 $ deactivate $ tox -e pep8 # should pass now I'd also like to know what would be the procedure to update sample config when new version of oslo.messaging is released? Maybe we should pin requirements to specific version of oslo library and update that requirement along with the sample config once new version is released? -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Juno-3 released
I'm glad to announce Juno-3 release of Murano: https://launchpad.net/murano/+milestone/juno-3 We've implemented 8 blueprints and closed 50 bugs. Thanks a lot to everyone who worked on this milestone. All these efforts helped to significantly improve quality and test-coverage of the project! Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Repositoris re-organization
I'd suggest to reduce number of Murano repositories for several reasons: * All other OpenStack projects have a single repo per project. While this point might look like something not worth mentioning, it's really important: - unified project structure simplifies life for new developers. once they get familiar with one project, they can expect something similar from another project - unified project structure simplifies life for deployers. similar project structure simplifies packaging and deployment automation * Another important reason is to simplify gated testing. Just take a look at Solum layout [1], they have everything needed (contrib, functionaltests) to run dvsm job in a single repo. One simple job definition [2] allows to install Solum in DevStack and run Tempest tests for Solum. * As a side-effect, this approach will improve integrity of project components. Having murano-common in the same repo with other components will help to catch integration issues earlier. In an ideal world there will be only the following repos: - murano (api, common, conductor, docs, repository, tests) - python-muranoclient - murano-dashboard - murano-agent - puppet-murano (optional, but nice to have) [1] https://github.com/stackforge/solum [2] https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/jenkins_job_builder/config/solum.yaml Thanks, Ruslan On Thu, Feb 6, 2014 at 3:14 PM, Serg Melikyan wrote: > Hi, Alexander, > > In general I am completely agree with Clint and Robert, and as one of > contributors of Murano I don't see any practical reasons for repositories > reorganization. And regarding of your proposal I have a few thoughts that I > would like to share below: > > >This enourmous amount of repositories adds too much infrustructural > complexity > Creating a new repository is a quick, easy and completely automated > procedure that requires only simple commit to Zuul configuration. All > infrastructure related to repositories is handled by Openstack CI and > supported by Openstack Infra Team, and actually don't require anything from > project development team. About what infrastructure complexity you are > talking about? > > >I actually think keeping them separate is a great way to make sure you > have ongoing API stability. (c) Clint > I would like to share a little statistic gathered by Stan Lagun > a little time ago regarding repositories count in different PaaS solution. > If you are concerned about large number of repositories used by Murano, you > will be quite amused: > >- https://github.com/heroku - 275 >- https://github.com/cloudfoundry - 132 > - https://github.com/openshift - 49 >- https://github.com/CloudifySource - 46 > > >First of all, I would suggest to have a single reposository for all the > three main components of Murano: main murano API (the contents of the > present), workflow execution engine (currently murano-conductor; also it > was suggested to rename the component itself to murano-engine for more > consistent naming) and metadata repository (currently murano-repository). > > *murano-api* and *murano-repository* have many things in common, they are > both present HTTP API to the user, and I hope would be rewritten to common > framework (Pecan?). But *murano-conductor* have only one thing in common > with other two components: code shared under *murano-common*. That > repository may be eventually eliminated by moving to Oslo (as it should be > done). > > >Also, it has been suggested to move our agents (both windows and unified > python) into the main repository as well - just to put them into a separate > subfolder. I don't see any reasons why they should be separated from core > Murano: I don't believe we are going to have any third-party > implementations of our "Unified agent" proposals, while this possibility > was the main reason for separatinng them. > > Main reason for murano-agent to have separate repository was not a > possibility to have another implementation, but that all sources that > should be able to be built as package, have tests and can be uploaded to > PyPI (or any other gate job) should be placed in different repository. > OpenStack CI have several rules regarding how repositories should be > organized to support running different gate jobs. For example, to run tests > *tox.ini* is need to be present in root directory, to build package > *setup.py* should be present in root directory. So we could not simply > move them to separate directories in main repository and have same > capabilities as in separate repository. > > >Next, deployment scripts and auto-generated docs: are there reasons why > they should be in their own repositories, instead of "docs" and > "tools/deployment" folders of the primary repo? I would prefer the latter: > docs and deployment scripts have no meaning without the sources which they > document/deploy - so it is better to have them consistent. > We have *developers documentation* alongside
[openstack-dev] [savanna] Documentation update for IceHouse
As it was decided on our latest meeting I've added a new blueprint for documentation updates: https://blueprints.launchpad.net/savanna/+spec/update-docs-icehouse Here is a list of work items I've proposed: * Update installation guide * Setup development environment * Devstack guide * Tempest tests for Savanna * DIB instructions * Replace github with git.o.o * How-to add a new migration script * Update rest-api docs Please, feel free to add new work items or assign them to yourself. Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] - Install Guide
Murano installation options are described at: http://murano.readthedocs.org/en/latest/ Thanks, Ruslan On Tue, Dec 2, 2014 at 1:41 PM, wrote: > Hi All, > > > > I am new to Murano and trying to integrate with Openstack Juno. > > Any build guides or help would be appreciated. > > > > Warm Regards, > > Raghavendra Lad > IDC–IC-P-Capability > > Infrastructure Consulting, > > Infrastructure Services – Accenture Operations > > Mobile - +91 9880040919 > > > > > > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise confidential information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the e-mail by you is prohibited. Where allowed by > local law, electronic communications with Accenture and its affiliates, > including e-mail and instant messaging (including content), may be scanned > by our systems for the purposes of information security and assessment of > internal compliance with Accenture policy. > __ > > www.accenture.com > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Murano Agent
On Mon, Dec 15, 2014 at 7:10 AM, wrote: > Please let me kow why Murano-agent is required and the components that needs > to be installed in it. You can find more details about murano agent at: https://wiki.openstack.org/wiki/Murano/UnifiedAgent It can be installed with diskimage-builder: http://git.openstack.org/cgit/stackforge/murano-agent/tree/README.rst#n34 - Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Nominating Kate Chernova for murano-core
Great addition to core team! +2 On Thu, Dec 25, 2014 at 11:02 AM, Timur Sufiev wrote: > +1 from me. > > On Thu, Dec 25, 2014 at 10:28 AM, Serg Melikyan > wrote: >> >> I'd like to propose that we add Kate Chernova to the murano-core. >> >> Kate is active member of our community for more than a year, she is >> regular participant in our IRC meeting and maintains a good score as >> contributor: >> >> http://stackalytics.com/report/users/efedorova >> >> Please vote by replying to this thread. As a reminder of your options, +1 >> votes from 5 cores is sufficient; a -1 is a veto. >> -- >> Serg Melikyan, Senior Software Engineer at Mirantis, Inc. >> http://mirantis.com | smelik...@mirantis.com >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > > -- > Timur Sufiev > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] SQLite support - drop or not?
On Fri, Jan 23, 2015 at 9:04 PM, Andrew Pashkin wrote: > Hello! > > Current situation with SQLite support: > - Migration tests does not run on SQLIte. > - At the same time migrations themselves support SQLite (with bugs). > > Today I came across this bug: > Error during execution of database downgrade > > We can resolve this bug by hardening SQLite support, in that case: > - We need to fix migrations and make them support SQLite without bugs, and > then continuously make some effort to maintain this support (manually > writing migrations and test cases for them). > - Also need to introduce migration tests run on SQLite. > > We also can drop SQLite support and in this case: > - We just factor out all that related to SQLite from migrations one time and > set this bug as "Won't fix". > > Let's discuss that. I agree that we don't need to support SQLite in migrations. As it was already said in [1], there is no point in running DB migrations against SQLite. Here is what I suggest to do: 1. Use ModelsMigrationsSync from [2] in tests to make sure that SQLAlchemy models are in sync with migrations. Usage example can be found at [3] 2. Populate DB schema from SQLAlchemy models in unit-tests which require access to DB 3. Wipe out everything related to SQLite from DB migrations code 4. Recommend all developers to use MySQL when they run Murano locally 5. For those who still insist on SQLite we can provide a command line option which would generate database schema from SQLAlchemy metadata. This should be declared as development only feature, not supported for any kind of production deployments [1] http://lists.openstack.org/pipermail/openstack-dev/2015-January/055058.html [2] http://git.openstack.org/cgit/openstack/oslo.db/tree/oslo_db/sqlalchemy/test_migrations.py [3] http://git.openstack.org/cgit/openstack/sahara/tree/sahara/tests/unit/db/migration/test_migrations_base.py#n198 Thanks, Ruslan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] SQLite support - drop or not?
On Mon, Jan 26, 2015 at 3:03 PM, Andrew Pashkin wrote: > On 23.01.2015 23:39, Ruslan Kamaldinov wrote: > > 1. Use ModelsMigrationsSync from [2] in tests to make sure that SQLAlchemy > models are in sync with migrations. Usage example can be found at [3] > > Seems like it is a great helper, as I understand it runs all migrations and > then compares DB state with models state and throws an error if they are out > of sync. > What I don't understand - why they still manually write checks for every > migration [1]? This is redundant, because ModelsMigrationsSync already does > the job testing that DB is in sync with models. > > By the way in Heat project they do the same thing [2]. What am I missing? I think it's still important to perform migration specific checks. We want to make sure that DB is in expected state after each specific migration. > [1] > http://git.openstack.org/cgit/openstack/sahara/tree/sahara/tests/unit/db/migration/test_migrations.py#n402 > [2] > https://github.com/openstack/heat/blob/7d4c4030c591ef5994db4327d66d353ad83c6ea8/heat/tests/db/test_migrations.py#L288 > 2. Populate DB schema from SQLAlchemy models in unit-tests which require > access to DB > > You mean - using these tools [3]? > > [3] > http://docs.sqlalchemy.org/en/rel_0_9/core/metadata.html#creating-and-dropping-database-tables Yes, this one. We still have this code in source tree: http://git.openstack.org/cgit/stackforge/murano/tree/murano/db/models.py#n289 > In what conditions 5) will fail? I see only these cases: > - If data migrations would be introduced and Murano would require some data > in DB to work correctly. Actually, we already do that. We populate initial set of categories: http://git.openstack.org/cgit/stackforge/murano/tree/murano/db/migration/alembic_migrations/versions/001_inital_version.py#n40 Would it affect anyone? I don't think so. You always can populate these categories manually. > - If Murano would use some database-specific features (stored procedures > etc). That should never happen :) > There are good chances that these cases will never happen in reality, as I > understand, so I tend to agree. Agree -- Ruslan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] SQLite support - drop or not?
On Mon, Jan 26, 2015 at 6:12 PM, Andrew Pashkin wrote: > On 26.01.2015 18:05, Ruslan Kamaldinov wrote: > > I think it's still important to perform migration specific checks. We want > to make sure that DB is in expected state after each specific migration. > > Why? 1. It's not just the schema we care about. It's the effect of particular DB migration script on data stored in DB. We need to make sure that data is not corrupted or lost in any way 2. Some migrations add or remove indexes and constraints, we might want to test that Thanks, Ruslan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How-to and best practices for developing new OpenStack service
Hi Ruslan! I can recommend two resources: 1. cookiecutter [1] - template for a new OpenStack project 2. http://ci.openstack.org/stackforge.html - this page helps to setup a project in OpenStack infrastructure, including code-review, Jenkins automation, etc. [1] http://git.openstack.org/cgit/openstack-dev/cookiecutter/tree/README.rst Regards, Ruslan On Wed, May 7, 2014 at 4:08 PM, Ruslan Kiianchuk wrote: > Hello, community. > > There are numerous open projects evolving that are designed to work with > OpenStack and follow OpenStack development principles (technologies stack, > architecture, etc.). There are even more efforts within many companies > working with cloud technologies. > > However it is hard to quick-start with all the practices developed at > OpenStack community and start developing an OpenStack service the right way. > > So are there any resources, tutorials, guides on how to start an OpenStack > service from scratch? > Something that would describe correct usage of Oslo project, common > architecture and give recommendations for developers. > > Thank you. > > -- > Sincerely, Ruslan Kiianchuk. > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How-to and best practices for developing new OpenStack service
On Thu, May 8, 2014 at 11:16 AM, Ruslan Kiianchuk wrote: > Short tutorial for developers not familiar with OpenStack infrastructure > still would be useful. My task is to help Python developers that haven't > worked with OpenStack before start an OpenStack service development in the > shortest possible period. If none will come up, I'll be thinking about > writing one :) Yes, intro tutorial could be useful. But before you start, please take a look at existing resources: 1. https://wiki.openstack.org/wiki/HowToContribute 2. https://wiki.openstack.org/wiki/GerritWorkflow 3. https://wiki.openstack.org/wiki/GitCommitMessages AFAIK Stefano and Tom are working on something similar. Added them to cc list. Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Release 0.5
I'd like to announce release of Murano 0.5 - Application Catalog for OpenStack. This release includes 28 implemented blueprints and 76 bug fixes. Release notes and tarballs can be found here: https://launchpad.net/murano/0.x/0.5 https://wiki.openstack.org/wiki/Murano/ReleaseNotes_v0.5 Here is the demo: https://drive.google.com/a/mirantis.com/file/d/0B3F5XCmYtnRdcnRUaXg0Q3BUcUU/edit More demos to come on the next week :) -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Heat] [Murano] [Solum] [TC] agenda for cross-project session - how to handle app lifecycle ?
I'd like to propose the following agenda for cross-project session "Solum, Murano, Heat: how to handle app lifecycle?": A number of projects are looking at going up the stack and handle application workload lifecycle management. This workshop aims at placing them all in the same room and creating opportunities for convergence in the future. 1. ~10-15 minutes to present Murano including answering questions. We will cover mission, positioning (relation to other OpenStack projects) and further roadmap which addresses concers on duplication of functionality with other projects. We've conducted a document [1], please append your questions or topics to discuss upfront. 2. ~10-15 minutes to present Solum, including Q&A. I would expect some overview of Solum too. Solum team, please expand your plan in the linked etherpad [2]. 3. Open discussion. Goal is to find a common ground on application lifecycle in OpenStack, determine project responsibilities (e.g Heat does X, Murano does Y, Solum does Z), discuss plan to eliminate any functionality duplication between these projects. If there is another project or initiative willing to present their work please let us know. I've added TC to the subject because we might need someone neutral and authoritative to moderate the session. Also, a number of questions were raised in comments to this session, I'll quote Steven Dake: > "I'd be interested in having some TC representation at this session to > understand how the TC would consider organizing these efforts (separate > programs or expand scope of orchestration program)." At least one TC member presence would be very welcome :) If you have a topic to add, please update session etherpad [2]. [1] https://etherpad.openstack.org/p/wGy9S94l3Q [2] https://etherpad.openstack.org/p/9XQ7Q2NQdv -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] [Murano] [Solum] [TC] agenda for cross-project session - how to handle app lifecycle ?
On Sat, May 10, 2014 at 7:02 AM, Ruslan Kamaldinov wrote: > I'd like to propose the following agenda for cross-project session > "Solum, Murano, Heat: how to handle app lifecycle?": > > A number of projects are looking at going up the stack and handle application > workload lifecycle management. This workshop aims at placing them all in the > same room and creating opportunities for convergence in the future. > > 1. ~10-15 minutes to present Murano including answering questions. We will >cover mission, positioning (relation to other OpenStack projects) and > further >roadmap which addresses concers on duplication of functionality with other >projects. We've conducted a document [1], please append your questions or >topics to discuss upfront. > > 2. ~10-15 minutes to present Solum, including Q&A. I would expect some >overview of Solum too. Solum team, please expand your plan in the linked >etherpad [2]. > > 3. Open discussion. Goal is to find a common ground on application lifecycle >in OpenStack, determine project responsibilities (e.g Heat does X, Murano >does Y, Solum does Z), discuss plan to eliminate any functionality >duplication between these projects. > > If there is another project or initiative willing to present their work please > let us know. > > I've added TC to the subject because we might need someone neutral and > authoritative to moderate the session. Also, a number of questions > were raised in > comments to this session, I'll quote Steven Dake: >> "I'd be interested in having some TC representation at this session to >> understand how the TC would consider organizing these efforts (separate >> programs or expand scope of orchestration program)." > > At least one TC member presence would be very welcome :) > > If you have a topic to add, please update session etherpad [2]. > > > [1] https://etherpad.openstack.org/p/wGy9S94l3Q > [2] https://etherpad.openstack.org/p/9XQ7Q2NQdv > > > -- > Ruslan I've updated the session etherpad [1] with the follow-up. Please see the bottom of the document (I didn't want to modify notes taken during the session). Session went well. There were couple of items/questions raised during the session which should be discussed in more details. As we agreed at the design session, let us leverage the fact of the local presence of the members of each team (Heat, Murano, Solum) and to have a follow up session to: - reiterate over the cross-projects session results and ensure everybody understands it the same way - to start discussing an approach to have a common cross-project environment description Zane expressed readiness to participate in such a session on Thursday. Let's figure out who else can participate from Heat and Solum team to have a meeting on Thursday in one of the design PODs at the second level. [1] https://etherpad.openstack.org/p/9XQ7Q2NQdv ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Meet Murano developers at Atlanta summit
We've seen a lot of interest in Murano at the Atlanta summit. In case if you would like to learn more, please stop by Mirantis booth (1st level) tomorrow (Thursday) from from 3PM to 5PM. Murano developers will be there and they will be happy to show you the demo in give more details. -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Meet Murano developers at Atlanta summit
On Wed, May 14, 2014 at 1:31 PM, Ruslan Kamaldinov wrote: > We've seen a lot of interest in Murano at the Atlanta summit. In case > if you would like to learn more, please stop by Mirantis booth (1st > level) tomorrow (Thursday) from from 3PM to 5PM. Murano developers > will be there and they will be happy to show you the demo in give more > details. > > -- > Ruslan Uh oh. Markeеplace on level 1 is closed today. We will be at the developer lounge in the B301-B306 area. Time is the same - from 3PM to 5PM on Thursday. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] repository murano-api renamed to murano
Just a heads up: Repository stackforge/murano-api was renamed to stackforge/murano on May 23. All the commit and review history is preserved and can be viewed under the new name [1], [2]. We also renamed [3]: * internal package name from muranoapi to murano * config file names under etc/murano Now, our devstack/tempest job is green again. As it was agreed on the meeting today we're going to mark it as voting in one week from now. [1] https://git.openstack.org/cgit/stackforge/murano [2] https://review.openstack.org/#/q/project:stackforge/murano,n,z [3] https://blueprints.launchpad.net/murano/+spec/rename-murano-api-to-murano -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Murano API improvements
Let's follow the standard procedure. Both blueprints lack specification of implementation details. There also has to be someone willing to implement these blueprints in near feature. I'm not opposed to these ideas and I'd really like to see Pecan added during Juno, but we still need to follow the procedure. I cannot approve an idea, it should be a specification. Let's work together on the new API specification first, then we'll need to find a volunteer to implement it on top of Pecan. -- Ruslan On Mon, Jun 2, 2014 at 8:35 AM, Timur Nurlygayanov wrote: > Hi all, > > We need to rewrite Murano API on new API framework and we have the commit: > https://review.openstack.org/#/c/60787 > (Sergey, sorry, but -1 from me, need to fix small isses) > > Also, today I created blueprint: > https://blueprints.launchpad.net/murano/+spec/murano-api-workers > this feature allows to run many API threads on one host and this allows to > scale Murano API processes. > > I suggest to update and merge this commit with migration to Pecan framework > and after that we can easily implement this blueprint and add many other > improvements to Murano API and Murano python agent. > > Ruslan, could you please approve these blueprints and target them to some > milestone? > > > Thank you! > > -- > > Timur, > QA Engineer > OpenStack Projects > Mirantis Inc > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TC] [Murano] Follow up on cross-project session
Hi community and TC members! First a little bit of history: Murano applied for incubation in February 2014 [1]. TC discussion [2] finished the following resolution (quote from ttx): "Murano is slightly too far up the stack at this point to meet the "measured progression of openstack as a whole" requirement. We'll facilitate collaboration in that space by setting up a cross-project session to advance this at the next design summit." Thanks to the TC, we had two official cross-project sessions at the Atlanta summit. We also had a follow-up session with a more focused group of people. - I would like to share the results of these sessions with the community and with the TC members. The official etherpads from the session is located here: https://etherpad.openstack.org/p/solum-murano-heat-session-results Now, more details. Here is the outcome of the first two (official) sessions: * We addressed questions about use-cases and user roles of Murano [3] * We clarified that Murano is already using existing projects and will use other projects (such as Heat and Mistral) to the fullest extent possible to avoid any functional duplication * We reached a consensus and vision on non-overlapping scopes of Murano and Solum Here is the link to the session etherpad [4]. Our follow-up session was very productive. The goal of this session was to define and document a clear scope for each project [5]. This is the main document in regards to the goals set by TC in response to Murano incubation request. We identified a clear scope for each project and possible points for integration and collaboration: * Application Development - Solum * Application Catalog/Composition - Murano (see https://etherpad.openstack.org/p/wGy9S94l3Q for details) * Application Deployment/Configuration - Heat We hope that we addressed major concerns raised at the TC meeting [2] initiated by the previous Murano incubation request. We will now focus on stabilizing and maturing the project itself with the end goal of again applying for incubation by the end of the Juno cycle. [1] http://lists.openstack.org/pipermail/openstack-dev/2014-February/027736.html [2] http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-03-04-20.02.log.txt [3] https://etherpad.openstack.org/p/wGy9S94l3Q [4] https://etherpad.openstack.org/p/9XQ7Q2NQdv [5] https://etherpad.openstack.org/p/solum-murano-heat-session-results Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TC] [Murano] Follow up on cross-project session
On Wed, Jun 11, 2014 at 8:07 PM, Thierry Carrez wrote: > I had a question about Murano's remaining feature set though. If the > application catalog lives in Glance and the application deployment lives > in Heat, what functionality does Murano end up providing ? Could you > elaborate on the "composition" mission ? Yes, application composition is the main feature of Murano from the user perspective. Let me outline how we see the project responsibilities. Murano will use Glance to store and manage application packages as Glance artifacts. Murano packages could have different formats - currently Murano package. TOSCA TCAR and APS packages [1] will be supported by Murano in the future. Glance artifacts API will allow to store, query, list and index packages. Murano will use Glance to store and manage application packages as Glance artifacts. Heat is used by Murano to manage OpenStack resources and software components. Heat engine is responsible for actual resource provisioning and deployment. On top of those Murano provides the following features: - allow users to combine various packages from catalog by using capabilities and requirements of applications - provide easy-to-use rich UI for end users who don’t necessarily have understanding of the underlying cloud infrastructure - Murano "knows" how to merge different packages and generates Heat template to deploy the environment, which in terms of Murano is a logical aggregation of multiple applications - as an application catalog allows app publishers and cloud owners to certify and license packages, provide additional partner information - allow to define billing rules. Murano can generate events predefined by app publisher to Ceilometer and integrate with 3rd party billing systems to bill users based on Ceilometer statistics - third-party services plumbing to support integration with APIs, both in stack, like Trove, and external Hopefully this document [2] started as initiative elaborated from our cross-project session will help to understand what Murano provides from user perspective. [1] http://doc.apsstandard.org/ [2] http://bit.ly/1lz3ULB Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] MuranoPL questions?
Joshua, Clint, The only platform I'm aware about which fully supports true isolation and which has been used in production for this purpose is Java VM. I know people who developed systems for online programming competitions and really smart kids tried to break it without any luck :) Since we're speaking about Heat, Mistral and Murano DSLs and all of them need an execution engine. Do you think that JVM could become a host for that engine? JVM has a lot of potential: - it allows to fine-tune security manager to allow/disallow specific actions - it can execute a lot of programming languages - Python, Ruby, JS, etc - it has been used in production for this specific purpose for years But it also introduces another layer of complexity: - it's another component to deploy, configure and monitor - it's non-python, which means it should be supported by infra - we will need to run java service and potentially have some java code to accept and process user code Thanks, Ruslan On Mon, Mar 17, 2014 at 10:40 PM, Joshua Harlow wrote: > So I guess this is similar to the other thread. > > http://lists.openstack.org/pipermail/openstack-dev/2014-March/030185.html > > I know that the way YQL has provided it could be a good example; where the > core DSL (the select queries and such) are augmented by the addition and > usage of JS, for example > http://developer.yahoo.com/yql/guide/yql-execute-examples.html#yql-execute-example-helloworld > (ignore that its XML, haha). Such usage already provides rate-limits and > execution-limits > (http://developer.yahoo.com/yql/guide/yql-execute-intro-ratelimits.html) and > afaik if something like what YQL is doing then u don't need to recreate > simialr features in your DSL (and then u also don't need to teach people > about a new language and syntax and ...) > > Just an idea (I believe lua offers similar controls/limits.., although its > not as popular of course as JS). > > From: Stan Lagun > > Reply-To: "OpenStack Development Mailing List (not for usage questions)" > > Date: Monday, March 17, 2014 at 3:59 AM > > To: "OpenStack Development Mailing List (not for usage questions)" > > Subject: Re: [openstack-dev] MuranoPL questions? > > Joshua, > > Completely agree with you. We wouldn't be writing another language if we > knew how any of existing languages can be used for this particular purpose. > If anyone suggest such language and show how it can be used to solve those > issues DSL was designed to solve we will consider dropping MuranoPL. np > > Surely DSL hasn't stood the test of time. It just hasn't had a chance yet. > 100% of successful programming languages were in such position once. > > Anyway it is the best time to come with your suggestions. If you know how > exactly DSL can be replaced or improved we would like you to share > > > On Wed, Mar 12, 2014 at 2:05 AM, Joshua Harlow > wrote: >> >> I guess I might be a bit biased to programming; so maybe I'm not the >> target audience. >> >> I'm not exactly against DSL's, I just think that DSL's need to be really >> really proven to become useful (in general this applies to any language that >> 'joe' comp-sci student can create). Its not that hard to just make one, but >> the real hard part is making one that people actually like and use and >> survives the test of time. That's why I think its just nicer to use >> languages that have stood the test of time already (if we can), creating a >> new DSL (muranoPL seems to be slightly more than a DSL imho) means creating >> a new language that has not stood the test of time (in terms of lifetime, >> battle tested, supported over years) so that's just the concern I have. >> >> Of course we have to accept innovation and I hope that the DSL/s makes it >> easier/simpler, I just tend to be a bit more pragmatic maybe in this area. >> >> Here's hoping for the best! :-) >> >> -Josh >> >> From: Renat Akhmerov >> Reply-To: "OpenStack Development Mailing List (not for usage questions)" >> >> Date: Monday, March 10, 2014 at 8:36 PM >> To: "OpenStack Development Mailing List (not for usage questions)" >> >> Subject: Re: [openstack-dev] MuranoPL questions? >> >> Although being a little bit verbose it makes a lot of sense to me. >> >> @Joshua, >> >> Even assuming Python could be sandboxed and whatever else that's needed to >> be able to use it as DSL (for something like Mistral, Murano or Heat) is >> done why do you think Python would be a better alternative for people who >> don't know neither these new DSLs nor Python itself. Especially, given the >> fact that Python has A LOT of things that they'd never use. I know many >> people who have been programming in Python for a while and they admit they >> don't know all the nuances of Python and actually use 30-40% of all of its >> capabilities. Even not in domain specific development. So narrowing a >> feature set that a language provides and limiting it to a certain domain >> vocabulary is what helps people solve tasks of that specific domain
Re: [openstack-dev] [Murano][Heat] MuranoPL questions?
Here is my 2 cents: I personally think that evolving Heat/HOT to what Murano needs for it's use cases is the best way to make PaaS layer of OpenStack to look and feel as a complete and fully integrated solution. Standardising these things in a project like TOSCA is another direction we all should follow. I think that TOSCA is the place where developers (like us), application developers and enterprises can collaborate to produce a common standard for application lifecycle management in the clouds. But before Murano contributors jump into direction of extending HOT to the goal of application (or system) lifecycle management, we need an agreement that this is the right direction for Heat/HOT/DSL and the Orchestration program. There are a lot of use cases that current HOT doesn't seem to be the right tool to solve. As it was said before, it's not a problem to collaborate on extending it those use cases. I'm just unsure if Heat team would like these use cases to be solved with Heat/HOT/DSL. For instance: - definition of an application which is already exposed via REST API. Think of something like Sahara (ex. Savanna) or Trove developed in-house for internal company needs. app publishers wouldn't be happy if they'll be forced to develop a new resource for Heat - definition of billing rules for an application If everyone agrees that this is the direction we all should follow, that we should expand HOT/DSL to that scope, that HOT should be the answer on "can you express it?", then awesome - we can start speaking about implementation details. If it's not the direction these projects should follow then at least finding where Heat ends and Murano starts to avoid any functionality duplication would be great. Thanks, Ruslan On Wed, Mar 19, 2014 at 2:07 AM, Keith Bray wrote: > Georgy, > > In consideration of the "can you express it" instead of the "who will > generate it," I see Heat's HOT evolving to support the expression of complex > multi-tier architectures and applications (I would argue you can already do > this today, perhaps with some additional features desired, e.g. Ability to > define cloud workflows and workflow execution rules which could come when we > have a workflow service like Mistral). Therefore, I would encourage Murano > contributors to consider whether they can help make Heat sufficiently cover > desired use cases. I have never viewed Heat templates as isolated > components of a multi-tier architecture. Instead, a single template or a > combination of master/subordinate templates together (using references, > nesting, or inclusion) could express the complete architecture, both > infrastructure and applications. > > If I've read your previous comments and threads correctly, you desire a way > to express System Lifecycle Management across multiple related applications > or components, whereby you view the System as a grouping of independently > developed and/or deployed (but systematically related) "components," whereby > you view Components as individual disconnected Heat templates that > independently describe different application stacks of the System. Did I > get that correct? If so, perhaps the discussion here is one of "scope" of > what can or should be expressed in a Heat template. Is it correct to state > that your argument is that a separate system (such as Murano) should be used > to express System Lifecycle Management as I've defined it here? If so, why > could we not use the Heat DSL to also define the System? The System > definition could be logically separated out into its own text file... But, > we'd have a common DSL syntax and semantics for both lower level and higher > level component interaction (a building block effect of sorts). > > As for "who will generate it," ( with "it" being the Heat multi-tier > application/infrastructure definition) I think that question will go through > a lot more evolution and could be any number of sources: e.g. Solum, Murano, > Horizon, Template Author with a text editor, etc. > > Basically, I'm a +1 for as few DSLs as possible. I support the position that > we should evolve HOT if needed vs. having two separate DSLs that are both > related to expressing application and infrastructure semantics. > > Workflow is quite interesting ... Should we be able to express imperative > workflow semantics in HOT? Or, should we only be able to declare workflow > configurations that get configured in a service like Mistral whereby > Mistral's execution of a workflow may need to invoke Heat hooks or Stack > Updates? Or, some other solution? > > I look forward to a design discussion on all this at the summit... This is > fun stuff to think about! > > -Keith > > From: Georgy Okrokvertskhov > > Reply-To: "OpenStack Development Mailing List (not for usage questions)" > > Date: Tuesday, March 18, 2014 1:49 PM > > To: "OpenStack Development Mailing List (not for usage questions)" > > Subject: Re: [openstack-dev] [Murano][Heat] MuranoPL questi
Re: [openstack-dev] Separating our Murano PL core in own package
On Mon, Mar 24, 2014 at 10:08 PM, Joshua Harlow wrote: > Seeing that the following repos already exist, maybe there is some need for > cleanup? > > - https://github.com/stackforge/murano-agent > - https://github.com/stackforge/murano-api > - https://github.com/stackforge/murano-common > - https://github.com/stackforge/murano-conductor > - https://github.com/stackforge/murano-dashboard > - https://github.com/stackforge/murano-deployment > - https://github.com/stackforge/murano-docs > - https://github.com/stackforge/murano-metadataclient > - https://github.com/stackforge/murano-repository > - https://github.com/stackforge/murano-tests > ...(did I miss others?) > > Can we maybe not have more git repositories and instead figure out a way to > have 1 repository (perhaps with submodules?) ;-) > > It appears like murano is already exploding all over stackforge which makes > it hard to understand why yet another repo is needed. I understand why from > a code point of view, but it doesn't seem right from a code organization > point of view to continue adding repos. It seems like murano > (https://github.com/stackforge/murano) should just have 1 repo, with > sub-repos (tests, docs, api, agent...) for its own organizational usage > instead of X repos that expose others to murano's internal organizational > details. > > -Josh Joshua, I agree that this huge number of repositories is confusing for newcomers. I've spent some time to understand mission of each of these repos. That's why we already did the cleanup :) [0] And I personally will do everything to prevent creation of new repo for Murano. Here is the list of repositories targeted for the next Murano release (Apr 17): * murano-api * murano-agent * python-muranoclient * murano-dashboard * murano-docs The rest of these repos will be deprecated right after the release. Also we will rename murano-api to just "murano". murano-api will include all the Murano services, functionaltests for Tempest, Devstack scripts, developer docs. I guess we already can update README files in deprecated repos to avoid further confusion. I wouldn't agree that there should be just one repo. Almost every OpenStack project has it's own repo for python client. All the user docs are kept in a separate repo. Guest agent code should live in it's own repository to keep number of dependencies as low as possible. I'd say there should be required/comfortable minimum of repositories per project. And one more nit correction: OpenStack has it's own git repository [1]. We shoul avoid referring to github since it's just a convinient mirror, while [1] is an official OpenStack repository. [0] https://blueprints.launchpad.net/murano/+spec/repository-reorganization [1] http://git.openstack.org/cgit/ Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano][Heat] MuranoPL questions?
Murano folks, I guess we should stop our debates with the Heat team about MuranoPL. What we should do instead is to carefully read this thread couple of times and collect and process all the feedback we've got. Steve and Zane did a very good job helping us to find a way to align with Heat and Orhcestration program. Let me outline the most important (imho) quotes: # START quotes Steven Dake: > I see no issue with HOT remaining simple and tidy focused entirely on > orchestration (taking a desired state and converting that into reality) with > some other imperative language layered on top to handle workflow and ALM. I > believe this separation of concerns is best for OpenStack and should be the > preferred development path. Zane Bitter: > I do think we should implement the hooks I mentioned at the start of this > thread to allow tighter integration between Heat and a workflow engine > (i.e. Mistral). > > So building a system on top of Heat is good. Building it on top of Mistral as > well would also be good, and that was part of the feedback from the TC. > > To me, building on top means building on top of the languages (which users > will have to invest a lot of work in learning) as well, rather than having a > completely different language and only using the underlying implementation(s). > > To me that implies that Murano should be a relatively thin wrapper that ties > together HOT and Mistral's DSL. Steve Dake: --- > I don't think HOT can do these things and I don't think we want HOT to do > these things. I am ok with that, since I don't see the pushback on having > two languages for two different things in OpenStack. I got from gokrove on > iRC today that the rationale for the pushback was the TC wanted Murano folks > to explore how to integrate better with Heat and possibly the orchestration > program. I don't see HOT as a place where there is an opportunity for scope > expansion. I see instead Murano creating HOT blobs and feeding them to Heat. Zane Bitter: > Because there have to be some base types that you can use as building blocks. > In the case of Heat, those base types are the set of things that you can > create by talking to OpenStack APIs authenticated with the user's token. > In the case of Mistral, I would expect it to be the set of actions that you > can take by talking to OpenStack APIs authenticated with the user's token. > And in the case of Murano, I would expect it to be the union of those two. > > > Everything is a combination of existing resources, because the set of existing > resources is the set of things which the operator provides as-a-Service. The > set of things that the operator provides as a service plus the set of things > that you can implement yourself on your own server (virtual or not) covers > the entire universe of things. What you appear to be suggesting is that > OpenStack must provide *Everything*-as-a-Service by allowing users to write > their own services and have the operator execute them as-a-Service. This > would be a breathtakingly ambitious undertaking, and I don't mean that in > a good way. > > When the TC said "Murano is slightly too far up the stack at this point to > meet the "measured progression of openstack as a whole" requirement", IMO > one of the major things they meant was that you're inventing your own > workflow thing, leading to duplication of effort between this and Workflow > as a Service. (And Mistral folks are in turn doing the same thing by not > using the same workflow library, taskflow, as the rest of OpenStack.) > Candidly, I'm surprised that this is not the #1 thing on your priority list > because IMO it's the #1 thing that will delay getting the project incubated. # END quotes Also I should quote what Georgy Okrokvertkhov said: > Having this aligned I see a Murano package as an archive with all necessary > definitions and resources and Murano service will just properly pass them to > related services like Heat and Mistral. I think then Murano DSL will be much > more simple and probably will be closer to declarative format with some > specific data operations. Summary: I would like to propose further path for Murano evolution where it evolves as an OpenStack project, aligned with others and developed in agreement between all the interested sides. Here is the plan: * Step forward and implement hooks (for workflow customization) in Heat. We will register a blueprint, discuss it with the Heat team and implement it * Use our cross-project session on ATL summit to set clear goals and expectations * Build pilot in Murano which: a. Uses new HOT Software components to do VM side software deployments b. Uses Mistral DSL to describe workflow. It'll require more focused discussion with Mistral team c. Bundles all that into an application package using new Murano DSL d. Allows users to combine applications in a single environenment * Continuously align with the He
Re: [openstack-dev] [Murano][Heat] MuranoPL questions?
On Thu, Mar 27, 2014 at 7:42 PM, Georgy Okrokvertskhov wrote: > Given that I don't see the huge overlap here with Murano functionality as > even if Solum stated that as a part of solution Heat template will be > generated it does not necessarily mean that Solum itself will do this. From > what is listed on the Solum page, in Solum sense - ALM is a way how the > application build from source promoted between different CI\CD environments > Dev, QA, Stage, Production. Solum can use other service to do this keeping > its own focus on the target area. Specifically to the environments - Solum > can use Murano environments which for Murano is just a logical unity of > multiple applications. Solum can add CI\CD specific stuff on top of it > keeping using Murano API for the environment management under the hood. > Again, this is a typical OpenStack approach to have different services > integrated to achieve the larger goal, keeping services itself very focused. Folks, I'd like to call for a cross-project work group to identify approaches for application description and management in the OpenStack cloud. As this thread shows there are several parties involved - Heat, Mistral, Murano, Solum (did I miss anyone?), there is no clear vision among us where and how we should describe things on top of Heat. We could spend another couple of months in debates, but I feel that focused group of dedicated people (i.e 2 from each project) would progress much faster and will be much more productive. What I'd suggest to expect from this joint group: * Identify how different aspects of applications and their lifecycle can be described and how they can coexist in OpenStack * Define a multi-layered structure to keep each layer with clear focus and set of responsibilities * End goal of the work for this group will be a document with a clear vision of covering areas higher up to Heat stack and how OpenStack should address that. This vision is not clear now for TC and that is the reason they say that it is to big step which Murano did * Agree on further direction * Come to ATL summit, agree again and drink beer Focused group would require additional communication channels: * Calls (Google Hangouts for instance) * Additional IRC meetings * Group work on design documents >From Murano project I'd like to propose the following participants: * Stan Lagun (sla...@mirantis.com) * Ruslan Kamaldinov (rkamaldi...@mirantis.com) Do colleagues from Heat, Solum and Mistral feel the same way and would like to support this movement and delegate their participants to this working group? Is this idea viable? -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Doc to with pointers on how to review?
On Mon, Mar 31, 2014 at 5:57 AM, Tom Fifield wrote: > Hi, > > Just wondering, do we have a document somewhere that educates people on how > to do a code review? eg giving a few pointers that reviewers for a > particular project normally look for Hi Tom, The closest to your description document is: https://wiki.openstack.org/wiki/ReviewChecklist ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Heat] [Murano] [Solum] applications in the cloud
This is a continuation of the "MuranoPL questions" thread. As a result of ongoing discussions, we figured out that definition of layers which each project operates on and has responsibility for is not yet agreed and discussed between projects and teams (Heat, Murano, Solum (in alphabetical order)). Our suggestion and expectation from this working group is to have such a definition of layers, agreement on it and an approach of reaching it. As a starting point, we suggest the following: There are three layers of responsibility: 1. Resources of the cloud 2. Applications of the cloud 3. Layers specific for Murano and Solum (to be further discussed and clarified, for this discussion it's out of scope) Layer 1 is obviously covered by Heat. Most of the disagreement is around layer 2. Our suggestion is to figure out the way where we can have common engine, DSL and approach to apps description. For this we'll take HOT and TOSCA as a basis and will work on addition of functionality missing from Murano and Solum point of view. We'll be happy if existing Heat team continue working on it, having the full control of the project, provided that we agree on functionality missing there from Murano and Solum point of view and addition of this functionality in a systematic way. Let me outline the main concern in regards to HOT from Murano perspective. Others concerns could be figured out later. The typical use case for Murano is the following: Application writer sets a requirement for her application to use RDBMS assuming that it can be MySQL, PostgreSQL or MSSQL. HOT engine should be able to understand such requirements and be able to satisfy them by instantiating instances with DB. End user should be able to control which particular type of DB should be used. This is quite similar to abstract requirements described in TOSCA. As Heat wants to cover TOSCA format too this Murano requirement is pretty well aligned with what HOT\TOSCA can provide. Hopefully this functionality can be introduced into HOT. Solum and Murano will leverage it by using some common API, and implementing layer 3 as thin as possible. Again, this is only suggested starting point, something to begin with. Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] team meeting April 3 1800 UTC [savanna]
On Wed, Apr 2, 2014 at 11:13 PM, Dmitry wrote: > Hi, > I'm wondering if you have plans to use Murano for a cluster management? > Thanks, > Dmitry Dmitry, Sahara is not going to use Murano for cluster management. Sahara uses Heat for underlying infrastructure management and various Hadoop management tools (e.g. Ambari from Hortonworks Data Platform) to manage Hadoop clusters. Also Sahara builds a nice layer of abstraction above different distributions to define and use cluster templates to provision Hadoop and related software. On the other hand, Murano might expose Sahara/Hadoop clusters in the app catalog. Sahara developers already work on implementation of Sahara resource in Heat. So, it'll be available through Heat templates. Here [0] you can find an example of Heat template for Sahara. [0] http://paste.openstack.org/show/64658/ Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Infra][Ceilometer][MagnetoDB] HBase database in devstack
On Tue, Apr 8, 2014 at 8:42 PM, Sean Dague wrote: > I think it's important to understand what we mean by "stable" in the > gate. It means that the end point is 99.% available. And that it's > up or down status is largely under our control. > > Things that are not stable by this definition which we've moved away > from for the gate: > * github.com - one of the reasons for git.openstack.org > * pypi.python.org - one of the reasons for our own pypi mirror > * upstream distro mirrors (we use cloud specific mirrors, which even > then do fail some times, more than we'd like) > > Fedora.org is not stable by this measure either. Downloading an iso from > fedora.org fails 5% of the time in the gate. > > I'm sure the Hortonworks folks are good folks, but by our standards of > reliability, no one stacks up. And an outage on their behalf means that > any project which gates on it will be blocked from merging any code > until it's addressed. If Ceilometer wants to take that risk in their > check queue (and be potentially blocked) that might be one thing, and we > could talk about that. But we definitely can't co-gate and block all of > openstack because of a hortonworks outage (which will happen, especially > if we download packages from them 600 - 1000 times a day). A natural solution for this would be a local to infra package mirror for HBase, Ceilometer, Mongo and all the dependencies not present in upstream Ubuntu. It seems straightforward from the technical point of view. It'll help to keep the Gate invulnerable to any outages in 3-rd party mirrors. Of course, someone has to signup to create scripts for that mirror and support it in the future. But, other concerns were expressed in the past. Let me quote Jeremy Stanley (from https://review.openstack.org/#/c/66884/): > This will need to be maintained in Ubuntu (and backported to 12.04 in Ubuntu > Cloud Archive or if necessary a PPA managed by the same package maintenance > team taking care of it in later Ubuntu releases). We don't install test > requirements system-wide on our long-running test slaves unless we can be > assured of security support from the Linux distribution vendor. There is no easy workaround here. Traditionally this kind of software is installed from vendor-supported mirrors and distributions. And they're the ones who maintain and provide security updates from Hadoop/HBase packages. In case of Ceilometer, I think that importance of having real tests on real databases is more important than the requirement for the packages to have security support from a Linux distribution. Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] [Murano] [Solum] applications in the cloud
Update: Stan filed a blueprint [0] for type interfaces in HOT. I would like to outline the current vision of Murano Application format, to make sure we're all on the same page. We had a valuable discussion in several MLs and we also had a lot of discussions between Murano team members. As a result of these discussion we see the following plan for Murano: * Adopt TOSCA for application definition in Murano. Work with TOSCA committee * Utilize Heat as much as possible. Participate in discussions and implementations for hooks mechanism in Heat. Participate in HOT format discussions * Adopt Mistral DSL for workflow management. Find a way to compose Heat templates and Mistral DSL * Keep MuranoPL engine in the source tree to take care of all the features we need for our users until those features are implemented on top things described above Murano, Heat teams, please let me know if this plan sounds sane to you. [0] https://blueprints.launchpad.net/heat/+spec/interface-types Thanks, Ruslan On Sat, Apr 5, 2014 at 12:25 PM, Thomas Spatzier wrote: > Clint Byrum wrote on 04/04/2014 19:05:04: >> From: Clint Byrum >> To: openstack-dev >> Date: 04/04/2014 19:06 >> Subject: Re: [openstack-dev] [Heat] [Murano] [Solum] applications inthe > cloud >> >> Excerpts from Stan Lagun's message of 2014-04-04 02:54:05 -0700: >> > Hi Steve, Thomas >> > >> > I'm glad the discussion is so constructive! >> > >> > If we add type interfaces to HOT this may do the job. >> > Applications in AppCatalog need to be portable across OpenStack clouds. >> > Thus if we use some globally-unique type naming system applications > could >> > identify their dependencies in unambiguous way. >> > >> > We also would need to establish relations between between interfaces. >> > Suppose there is My::Something::Database interface and 7 compatible >> > materializations: >> > My::Something::TroveMySQL >> > My::Something::GaleraMySQL >> > My::Something::PostgreSQL >> > My::Something::OracleDB >> > My::Something::MariaDB >> > My::Something::MongoDB >> > My::Something::HBase >> > >> > There are apps that (say SQLAlchemy-based apps) are fine with any >> > relational DB. In that case all templates except for MongoDB and HBase >> > should be matched. There are apps that design to work with MySQL only. > In >> > that case only TroveMySQL, GaleraMySQL and MariaDB should be matched. > There >> > are application who uses PL/SQL and thus require OracleDB (there can be >> > several Oracle implementations as well). There are also applications >> > (Marconi and Ceilometer are good example) that can use both some SQL > and >> > NoSQL databases. So conformance to Database interface is not enough and >> > some sort of interface hierarchy required. >> >> IMO that is not really true and trying to stick all these databases into >> one "SQL database" interface is not a use case I'm interested in >> pursuing. >> >> Far more interesting is having each one be its own interface which apps >> can assert that they support or not. > > Agree, this looks like a feasible goal and would already bring some value > add in that one could look up appropriate provider templates instead of > having to explicitly link them in environments. > Doing too much of inheritance sounds interesting at first glance but > burries a lot of complexity. > >> >> > >> > Another thing that we need to consider is that even compatible >> > implementations may have different set of parameters. For example >> > clustered-HA PostgreSQL implementation may require additional > parameters >> > besides those needed for plain single instance variant. Template that >> > consumes *any* PostgreSQL will not be aware of those additional > parameters. >> > Thus they need to be dynamically added to environment's input > parameters >> > and resource consumer to be patched to pass those parameters to actual >> > implementation >> > >> >> I think this is a middleware pipe-dream and the devil is in the details. >> >> Just give users the ability to be specific, and then generic patterns >> will arise from those later on. >> >> I'd rather see a focus on namespacing and relative composition, so that >> providers of the same type that actually do use the same interface but >> are alternate implementations will be able to be consumed. So for > instance >> there is the non-Neutron LBaaS and the Neutron LBaaS, and both have their >> merits for operators, but are basically identical from an application >> standpoint. That seems a better guiding use case than different > databases. >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing l
Re: [openstack-dev] [Murano] Working in devstack?
Hi Mark! Actually we have working Devstack scripts at [0]. We use these scripts to setup Murano in the dsvm (devstack/tempest) gate job which run Murano functional tests. It means that our Devstack scripts are in a good shape. Please use Devstack scripts from [0], all the other scripts are deprecated. I will make sure we mark them as deprecated ASAP. There is also a patch [1] which enables murano-dashboard in Devstack. It should be merged soon. What we're missing at this moment is developer documentation for the current release 0.5. I've added a blueprint [2] and started working on it. Feel free to ping us at #murano channel (my irc handle is ruhe) if you have any questions. [0] http://git.openstack.org/cgit/stackforge/murano-api/tree/contrib/devstack [1] https://review.openstack.org/#/c/87751/ [2] https://blueprints.launchpad.net/murano/+spec/murano-dev-doc-0.5 Thanks, Ruslan On Wed, Apr 16, 2014 at 3:53 AM, Mark Kirkwood wrote: > Thanks...yes, I'd not realized that running ./stack.sh again would unpatch > my murano-api/setup.sh! Rerunning the db setup as you suggested gives me the > tables. I didn't do it in as tidy a manner as yours however :-) > > $ export OS_USERNAME=admin > $ export OS_PASSWORD=swordfish > $ export OS_TENANT_NAME=demo > $ export OS_AUTH_URL=http://host:5000/v2.0/ > $ murano-manage --config-file /etc/murano/murano-api.conf db-sync > > Cheers > > Mark > > > On 16/04/14 11:23, Georgy Okrokvertskhov wrote: > > Hi Mark, > > Thank you for a detailed report. As I know Murano team is working on fixing > devstack scripts. > > As for DB setup it should be done by a command: tox -evenv -- murano-manage > --config-file etc/murano/murano-api.conf db-sync > > It works in my testing environment. > > Thanks > Georgy > > > On Tue, Apr 15, 2014 at 4:11 PM, Mark Kirkwood < > mark.kirkw...@catalyst.net.nz> wrote: > > Hi all, > > There is some interest here in making use of Murano for Samba ADDC a > service...so we've been (attempting) to get it up and running in devstack. > In the process I've managed to get myself confused about the correct > instructions for doing this: > > - the docs suggest http://murano-docs.github.io/latest/getting-started/ > content/ch04s03.html > - the project provides murano-deployment/devstack-scripts/README.rst) > > ...which are markedly different approaches (it *looks* like the project > README.rst is out of date, as it the scripts it runs try to get > heat-horizon from repos that do not exist anymore). If this approach is not > longer workable, we should really remove (or correct these instructions). > > So following http://murano-docs.github.io/latest/getting-started/ > content/ch04s03.html inside a Ubuntu 12.04 VM I stumbled into a few bugs: > > - several missing deps in various murano-*/requirements.txt (see attached) > - typo in /murano-api/setup.sh (db_sync vs db-sync) > > Fixing these seems to get most things going (some tabs crash with missing > tables, so I need to dig a bit more to find where they get created...). > > Any pointers/etc would be much appreciated! > > Cheers > > Mark > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [murano] Proposal to add Ruslan Kamaldinov to murano-core team
Thank you, guys :) Ruslan On Mon, Apr 21, 2014 at 1:40 PM, Timur Nurlygayanov wrote: > +1 > > Ruslan, congratulations! > > > > On Fri, Apr 18, 2014 at 1:41 PM, Timur Sufiev wrote: >> >> Ruslan, >> >> welcome to the Murano core team :)! >> >> On Thu, Apr 17, 2014 at 7:32 PM, Anastasia Kuznetsova >> wrote: >> > +1 >> > >> > >> > On Thu, Apr 17, 2014 at 7:11 PM, Stan Lagun wrote: >> >> >> >> +1 >> >> >> >> Sincerely yours, >> >> Stan Lagun >> >> Principal Software Engineer @ Mirantis >> >> >> >> >> >> >> >> On Thu, Apr 17, 2014 at 6:51 PM, Georgy Okrokvertskhov >> >> wrote: >> >>> >> >>> +1 >> >>> >> >>> >> >>> On Thu, Apr 17, 2014 at 6:01 AM, Dmitry Teselkin >> >>> >> >>> wrote: >> >>>> >> >>>> +1 >> >>>> >> >>>> Agree >> >>>> >> >>>> >> >>>> On Thu, Apr 17, 2014 at 4:51 PM, Alexander Tivelkov >> >>>> wrote: >> >>>>> >> >>>>> +1 >> >>>>> >> >>>>> Totally agree >> >>>>> >> >>>>> -- >> >>>>> Regards, >> >>>>> Alexander Tivelkov >> >>>>> >> >>>>> >> >>>>> On Thu, Apr 17, 2014 at 4:37 PM, Timur Sufiev >> >>>>> wrote: >> >>>>>> >> >>>>>> Guys, >> >>>>>> >> >>>>>> Ruslan Kamaldinov has been doing a lot of things for Murano >> >>>>>> recently >> >>>>>> (including devstack integration, automation scripts, making Murano >> >>>>>> more compliant with OpenStack standards and doing many reviews). >> >>>>>> He's >> >>>>>> actively participating in our ML discussions as well. I suggest to >> >>>>>> add >> >>>>>> him to the core team. >> >>>>>> >> >>>>>> Murano folks, please say your +1/-1 word. >> >>>>>> >> >>>>>> -- >> >>>>>> Timur Sufiev >> >>>>>> >> >>>>>> ___ >> >>>>>> OpenStack-dev mailing list >> >>>>>> OpenStack-dev@lists.openstack.org >> >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>>>> >> >>>>> >> >>>>> >> >>>>> ___ >> >>>>> OpenStack-dev mailing list >> >>>>> OpenStack-dev@lists.openstack.org >> >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Thanks, >> >>>> Dmitry Teselkin >> >>>> Deployment Engineer >> >>>> Mirantis >> >>>> http://www.mirantis.com >> >>>> >> >>>> ___ >> >>>> OpenStack-dev mailing list >> >>>> OpenStack-dev@lists.openstack.org >> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Georgy Okrokvertskhov >> >>> Architect, >> >>> OpenStack Platform Products, >> >>> Mirantis >> >>> http://www.mirantis.com >> >>> Tel. +1 650 963 9828 >> >>> Mob. +1 650 996 3284 >> >>> >> >>> ___ >> >>> OpenStack-dev mailing list >> >>> OpenStack-dev@lists.openstack.org >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >> >> >> >> >> >> ___ >> >> OpenStack-dev mailing list >> >> OpenStack-dev@lists.openstack.org >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> > >> > >> > ___ >> > OpenStack-dev mailing list >> > OpenStack-dev@lists.openstack.org >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> >> >> >> -- >> Timur Sufiev >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > > Timur, > QA Engineer > OpenStack Projects > Mirantis Inc > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] Design summit preparation - Next steps for Heat Software Orchestration
On Tue, Apr 22, 2014 at 8:42 PM, Thomas Spatzier wrote: > #2 Enable add-hoc actions on software components: > Apart from basic resource lifecycle hooks, it would be desirable to allow > for invocation of add-hoc actions on software. Examples would be the ad-hoc > creation of DB backups, application of patches, or creation of users for an > application. Such hooks (implemented as scripts, Chef recipes or Puppet > facts) could be defined in the same way as basic lifecycle hooks. They > could be triggered by doing property updates on the respective > SoftwareDeployment resources (just a thought and to be discussed during > design sessions). > I think this item could help bridging over to some discussions raised by > the Murano team recently (my interpretation: being able to trigger actions > from workflows). It would add a small feature on top of the current > software orchestration in Heat and keep definitions in one place. And it > would allow triggering by something or somebody else (e.g. a workflow) > probably using existing APIs. Hi Thomas, This is exactly what we need in Heat for Murano workflows. Also, I believe that what you described above, perfectly maps on TOSCA, which is also very good for us :) >From the implementation point of view and our previous discussions I figure out that Mistral can be a good fit for lifecycle hooks execution. Renat (Mistral lead) added a topic [1] for Heat weekly meeting, I hope we can discuss this today as part of that topic. [1] https://wiki.openstack.org/wiki/Meetings/HeatAgenda Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] weekly meeting cancelled
Since there are no major updates and some folks will not be able to join Murano weekly meeting, we've decided to skip it for today. Short update: 1. The only remaining major item left for juno is documentation and everyone is pretty aware about items we need to have in our docs: https://etherpad.openstack.org/p/murano-docs 2. We were not able to merge trusts support in Juno and had to postpone it to Kilo 3. A few bugs need to be fixed before the release, most of them have patches on review 4. Murano 3rd party CI wasn't quite stable for a few days due relocation to another DC. Today it got back to normal state Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Juno RC2 available
We had to release RC2 to address an issue blocking the inclusion of Murano Dashboard in Debian experimental packages. Tarballs with RC2 are available at: https://launchpad.net/murano/juno/juno-rc2 Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Murano 2014.2 "Juno" is released
I'm glad to announce release of Murano 2014.2 code-named "Juno". This release includes 39 implemented blueprints and 140 bugfixes. Source tarballs along with detailed list of features and bugfixes can be found at the following link: https://launchpad.net/murano/+milestone/2014.2 Release notes: https://wiki.openstack.org/wiki/Murano/ReleaseNotes/Juno Thanks to everyone who participated in development of Murano! PS: special thanks to Sergey Lukjanov for handling the release process. -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] IRC meeting cancelled today
Folks, I'm canceling our weekly IRC meeting today since we've just released Murano 2014.2 and still working on figuring out roadmap for Kilo cycle. Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TC] [Murano] Follow up on cross-project session
Hi Thierry! On Wed, Jun 18, 2014 at 1:14 PM, Thierry Carrez wrote: > So to take a practical example, Murano lets you pick (using UI or CLI) a > wordpress package (which requires a DB) and compose it with a mysql > package (which provides a DB), and will deploy that composition using > Heat ? And additionally, it provides package-publisher-friendly features > like certification, licensing and bulling ? That's a correct example. > Does that mean, to come back to my example above, that we could substitute a > Trove resource to the mysql package? Yes. Catalog may have several packages for MySQL implementations: MySQL Galera, single node, Trove-based. All of them can return database connection string which will be used by wordpress. > or put a Neutron LBaaS load balancer on top ? It should be possible if application can run behind a load balancer and there is an LBaaS resource in Heat. > or publish a DNS entry via Designate ? Should be possible, given the underlying infrastructure and catalog provides needed resources. As Murano uses Heat under the hood, whatever is possible in Heat will be possible in Murano. Once Designate resource is available in Heat, application developers can use it. Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Nominations to Murano core
I would like to nominate Serg Melikyan and Steve McLellan to Murano core. Serge has been a significant reviewer in the Icehouse and Juno release cycles. Steve has been providing consistent quality reviews and they continue to get more frequent and better over time. Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Nominations to Murano core
Serg, Steve, welcome to Murano core team! -- Ruslan On Thu, Jun 26, 2014 at 8:06 PM, Georgy Okrokvertskhov wrote: > +1 for both! Thanks for the great work! > > Regards, > Gosha > > > On Thu, Jun 26, 2014 at 7:05 AM, Timur Sufiev wrote: >> >> +1 for both. >> >> On Thu, Jun 26, 2014 at 3:29 PM, Stan Lagun wrote: >> > +1 for both (or should I say +2?) >> > >> > Sincerely yours, >> > Stan Lagun >> > Principal Software Engineer @ Mirantis >> > >> > >> > >> > On Thu, Jun 26, 2014 at 1:49 PM, Alexander Tivelkov >> > >> > wrote: >> >> >> >> +1 on both Serge and Steve >> >> >> >> -- >> >> Regards, >> >> Alexander Tivelkov >> >> >> >> >> >> On Thu, Jun 26, 2014 at 1:37 PM, Ruslan Kamaldinov >> >> wrote: >> >>> >> >>> I would like to nominate Serg Melikyan and Steve McLellan to Murano >> >>> core. >> >>> >> >>> Serge has been a significant reviewer in the Icehouse and Juno release >> >>> cycles. >> >>> >> >>> Steve has been providing consistent quality reviews and they continue >> >>> to get more frequent and better over time. >> >>> >> >>> >> >>> Thanks, >> >>> Ruslan >> >>> >> >>> ___ >> >>> OpenStack-dev mailing list >> >>> OpenStack-dev@lists.openstack.org >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> >> >> >> >> ___ >> >> OpenStack-dev mailing list >> >> OpenStack-dev@lists.openstack.org >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> > >> > >> > ___ >> > OpenStack-dev mailing list >> > OpenStack-dev@lists.openstack.org >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> >> >> >> -- >> Timur Sufiev >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > Georgy Okrokvertskhov > Architect, > OpenStack Platform Products, > Mirantis > http://www.mirantis.com > Tel. +1 650 963 9828 > Mob. +1 650 996 3284 > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] [Glance] Image tagging
I was going to refer to Graffiti as a longer term plan for image tagging. This initiative seems to be like a really good fit for our image tagging use-cases. For a short-term solutions proposed by Steve I have a couple of comments: > 1) Store allowed tags in the database, and allow administrators to add > to that list. Ordinary users would likely not be able to create tags, though > they could use pre-defined ones for images they owned. In some cases users might upload their own packages and they would likely need to mark some images as compatible with those specific packages. But I think there is a solution. Each time a new package is uploaded, Murano could create a tag with the same name (or fqn). That could help users to to tag package-specific images. > 2) Have some public tags, but also allow user-specified tags for > private packages. I think this leads to all sorts of tricky edge cases Agree, edge cases will bring a lot of unnecessary complexity. > 3) Allow freeform tags (i.e. don’t provide any hints). Since there’s no > formal link between the tag that a package looks for and the tags currently > defined in code, this wouldn’t make anything more susceptible to > inaccuracies In general, I tend to agree that option 1 (with a slight modification) is a good fit for a short-term solution. Thanks, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [savanna] Savanna usability
Erik, Most of these issues have corresponding blueprints/bugs filed in Launchpad: 1. https://bugs.launchpad.net/savanna/+bug/1204943 2. https://blueprints.launchpad.net/savanna/+spec/modifying-cluster-template https://blueprints.launchpad.net/savanna/+spec/modifying-node-group-template 3. https://bugs.launchpad.net/savanna/+bug/1174612 4. https://bugs.launchpad.net/savanna/+bug/1183387 I agree that these are really usability issues and need to be resolved eventually. Issue #4 is the most complicated. As a first step we need to implement common code which would gather detailed cluster status with info about each node. Once it's done we'll be able to apply policies you're suggesting. Ruslan On Monday, September 16, 2013 at 8:43 PM, Erik Bergenholtz wrote: > I want to start a dialog around adding some usability features to Savanna, > now that I have had a chance to spend a fair amount of time provisioning, > scaling and changing clusters. Here is a list of items that I believe are > important to address; comments welcomed: > 1. Changing an OpenStack flavor associated with a node-group template has a > ripple effect on both node group and cluster templates. The reason for this > is that OpenStack does not support modification of flavors; when a flavor is > modified (RAM, CPU, Root Disk, etc.) the flavor is deleted and a new one is > created resulting in a new flavor id. The implication is that both node-group > referencing the flavor [id] and any cluster templates referencing the > affected node-group become stale and unusable. A user then has to start from > scratch, creating new node-groups and cluster templates for a simple flavor > change. > a. A possible solution to this is to internally associate flavor name with > node-group and look up the flavor id based on flavor name when provisioning > instances > b. At a minimum it should be possible to change the flavor id associated with > a node-group. See #2. > > 2. Cluster templates and node group templates are immutable. This is more of > an issue at the node-group level as I often times want to make changes to a > node-group and want to affect all cluster templates that make use of that > node-group. I see this as being fairly commonplace. > > 3. Before provisioning a cluster, quotas should be checked to make sure that > enough quota exists. I know this can be done transactionally (check quota, > spawn cluster), but a basic check would go a long way. > > 4. Spawning a large cluster comes with some problems today, as Savanna will > abort if a single VM fails. In deploying large clusters (100’s to x1000), > which will be commonplace, having a single slave VM (i.e. data node) not > spawn properly should not necessarily be a reason to abort the entire > deployment, in particular in a scaling operation. Obviously the failing VM > cannot be a master service. This applies both to the plugin as well as > controller as they are both involved in the deployment. I could see a > possible solution being the creation of an error/fault policy allowing a user > to specify (perhaps optionally) a % or hard number for minimum # of nodes > that need to come up without aborting deployment. > a. This also applies to scaling the cluster by larger increments > > Just some thoughts based on my experience last week; comments welcomed. > > Best, > > Erik > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader of > this message is not the intended recipient, you are hereby notified that any > printing, copying, dissemination, distribution, disclosure or forwarding of > this communication is strictly prohibited. If you have received this > communication in error, please contact the sender immediately and delete it > from your system. Thank You. > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org (mailto:OpenStack-dev@lists.openstack.org) > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [savanna] neutron and private networks
This approach looks good. But I'd like to outline an issue people might hit in real-world production installations: Let's imagine OpenStack is installed in HA mode. There are several controllers, they run Savanna and Q3 services in HA mode. The problem is that active Savanna service should run on the same host with active Q3 service. Otherwise proposed approach will not work. It will require some advanced configuration in HA software - Savanna should always run on the same host with Q3. Ruslan On Thu, Oct 3, 2013 at 7:21 PM, Jon Maron wrote: > Hi, > > I'd like to raise an issue in the hopes of opening some discussion on > the IRC chat later today: > > We see a critical requirement to support the creation of a savanna > cluster with neutron networking while leveraging a private network (i.e. > without the assignment of public IPs) - at least during the provisioning > phase. So the current neutron solution coded in the master branch appears > to be insufficient (it is dependent on the assignment of public IPs to > launched instances), at least in the context of discussions we've had with > users. > > We've been experimenting and trying to understand the viability of such > an approach and have had some success establishing SSH connections over a > private network using paramiko etc. So as long as there is a mechanism to > ascertain the namespace associated with the given cluster/tenant > (configuration? neutron client?) it appears that the modifications to the > actual savanna code for the instance remote interface (the SSH client code > etc) will be fairly small. The namespace selection could potentially be > another field made available in the dashboard's cluster creation interface. > > -- Jon > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Savanna] Documentation for the upcoming release 0.3
Folks, I've created a blueprint and added several work items: https://blueprints.launchpad.net/savanna/+spec/docs-03 Feel free to add more items to this blueprint. -- Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Savanna-all] installation problem
It seems that you're using the latest code from Savanna v0.2, but following instructions from Savanna v0.1. Please follow these docs: https://savanna.readthedocs.org/en/latest/ Ruslan On Friday, July 12, 2013 at 1:19 PM, Arindam Choudhury wrote: > Hi, > > While installing savanna I am getting this error: > > [arindam@sl-3 savanna]$ tox -evenv -- bin/savanna-db-manage --config-file > etc/savanna/savanna.conf reset-db --with-gen-templates > GLOB sdist-make: /home/arindam/savanna/setup.py > venv inst-nodeps: /home/arindam/savanna/.tox/dist/savanna-0.2.a12.gf85d74f.zip > venv runtests: commands[0] | bin/savanna-db-manage --config-file > etc/savanna/savanna.conf reset-db --with-gen-templates > ERROR: InvocationError: could not find executable 'bin/savanna-db-manage' > _ summary > __ > ERROR: venv: commands failed > > > Regards, > Arindam > -- > Mailing list: https://launchpad.net/~savanna-all > Post to : savanna-...@lists.launchpad.net > (mailto:savanna-...@lists.launchpad.net) > Unsubscribe : https://launchpad.net/~savanna-all > More help : https://help.launchpad.net/ListHelp > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Savanna-all] installation problem
That's ok. You need to specify tenant id and provide auth token in headers. You can find detailed how-tos here https://savanna.readthedocs.org/en/latest/devref/quickstart.html btw, please use openstack-dev mail-list for Savanna-related questions. Just prefix mail subject with [Savanna]. Ruslan On Friday, July 12, 2013 at 1:59 PM, Arindam Choudhury wrote: > Hi, > > The new document works great except this command $ sudo pip install > savannadashboard. > > I changed the savanna config to: > [DEFAULT] > > # REST API config > port=8386 > allow_cluster_ops=true > > # Address and credentials that will be used to check auth tokens > os_auth_host=192.168.122.11 > os_auth_port=35357 > os_admin_username=admin > os_admin_password=openstack > os_admin_tenant_name=admin > > # (Optional) Name of log file to output to. If not set, > # logging will go to stdout. (string value) > log_file=/home/arindam/savanna.log > > [cluster_node] > > # An existing user on Hadoop image (string value) > #username=root > > # User's password (string value) > #password=swordfish > > # When set to false, Savanna uses only internal IP of VMs. > # When set to true, Savanna expects OpenStack to auto-assign > # floating IPs to cluster nodes. Internal IPs will be used for > # inter-cluster communication, while floating ones will be > # used by Savanna to configure nodes. Also floating IPs will > # be exposed in service URLs (boolean value) > use_floating_ips=true > > [sqlalchemy] > > # URL for sqlalchemy database (string value) > database_uri=sqlite:tmp/savanna-server.db > > > and I changed config of dashboard as specified. > > but I can not access the savanna dashboard: > > [arindam@sl-3 ~]$ curl http://localhost:8386/v1.0 > > > 401 Unauthorized > > > 401 Unauthorized > This server could not verify that you are authorized to access the document > you requested. Either you supplied the wrong credentials (e.g., bad > password), or your browser does not understand how to supply the credentials > required. > Authentication required > > > > > > > From: arin...@live.com (mailto:arin...@live.com) > To: rkamaldi...@mirantis.com (mailto:rkamaldi...@mirantis.com) > Subject: RE: [Savanna-all] installation problem > Date: Fri, 12 Jul 2013 11:33:24 +0200 > > Thanks. > > Date: Fri, 12 Jul 2013 13:25:45 +0400 > From: rkamaldi...@mirantis.com (mailto:rkamaldi...@mirantis.com) > To: arin...@live.com (mailto:arin...@live.com) > CC: openstack-dev@lists.openstack.org > (mailto:openstack-dev@lists.openstack.org); savanna-...@lists.launchpad.net > (mailto:savanna-...@lists.launchpad.net) > Subject: Re: [Savanna-all] installation problem > > It seems that you're using the latest code from Savanna v0.2, but following > instructions from Savanna v0.1. > Please follow these docs: https://savanna.readthedocs.org/en/latest/ > > > Ruslan > > On Friday, July 12, 2013 at 1:19 PM, Arindam Choudhury wrote: > > > Hi, > > > > While installing savanna I am getting this error: > > > > [arindam@sl-3 savanna]$ tox -evenv -- bin/savanna-db-manage --config-file > > etc/savanna/savanna.conf reset-db --with-gen-templates > > GLOB sdist-make: /home/arindam/savanna/setup.py > > venv inst-nodeps: > > /home/arindam/savanna/.tox/dist/savanna-0.2.a12.gf85d74f.zip > > venv runtests: commands[0] | bin/savanna-db-manage --config-file > > etc/savanna/savanna.conf reset-db --with-gen-templates > > ERROR: InvocationError: could not find executable 'bin/savanna-db-manage' > > _ summary > > __ > > ERROR: venv: commands failed > > > > > > Regards, > > Arindam > > -- > > Mailing list: https://launchpad.net/~savanna-all > > Post to : savanna-...@lists.launchpad.net > > (mailto:savanna-...@lists.launchpad.net) > > Unsubscribe : https://launchpad.net/~savanna-all > > More help : https://help.launchpad.net/ListHelp > > > > > > > > -- > Mailing list: https://launchpad.net/~savanna-all > Post to : savanna-...@lists.launchpad.net > (mailto:savanna-...@lists.launchpad.net) > Unsubscribe : https://launchpad.net/~savanna-all > More help : https://help.launchpad.net/ListHelp > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Savanna-all] How to run hadoop jobs
Hi Arindam,
There was a validation error during cluster creation. Can you send contents of
cluster_create.json ?
Also, please attach output of:
$ http $SAVANNA_URL/images X-Auth-Token:$AUTH_TOKEN
$ http $SAVANNA_URL/node-group-templates X-Auth-Token:$AUTH_TOKEN
$ http $SAVANNA_URL/cluster-templates X-Auth-Token:$AUTH_TOKEN
Savanna debug logs also would be helpful.
Please use *openstack-dev* as a mail-list for Savanna-related questions. Just
prefix the subject with [Savanna].
Thanks,
Ruslan
On Saturday, July 13, 2013 at 9:24 PM, Arindam Choudhury wrote:
> Hi,
>
> I installed savanna from the git repository. I can create cluster but only
> with admin, when I try to create cluster with non-admin user it gives me an
> error:
>
> (keystone_arindam)]# http $SAVANNA_URL/clusters X-Auth-Token:$AUTH_TOKEN <
> cluster_create.json
> HTTP/1.1 500 INTERNAL SERVER ERROR
> Content-Length: 111
> Content-Type: application/json
> Date: Sat, 13 Jul 2013 16:47:25 GMT
>
> {
> "error_code": 500,
> "error_message": "Error occurred during validation",
> "error_name": "INTERNAL_SERVER_ERROR"
> }
>
> The hadoop daemons are not started automatically so I could not run the test
> job.
>
> hadoop@cluster-1-master-001:/usr/share/hadoop$ hadoop jar hadoop-examples.jar
> pi 10 100
> Exception in thread "main" java.io.IOException: Error opening job jar:
> hadoop-examples.jar
> at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
> Caused by: java.io.FileNotFoundException: hadoop-examples.jar (No such file
> or directory)
> at java.util.zip.ZipFile.open(Native Method)
> at java.util.zip.ZipFile.(ZipFile.java:214)
> at java.util.zip.ZipFile.(ZipFile.java:144)
> at java.util.jar.JarFile.(JarFile.java:153)
> at java.util.jar.JarFile.(JarFile.java:90)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
> hadoop@cluster-1-master-001:/usr/share/hadoop$ start-all.sh
> (http://start-all.sh)
> mkdir: cannot create directory `/mnt/log': Permission denied
> chown: cannot access `/mnt/log/hadoop/hadoop/': No such file or directory
> starting namenode, logging to
> /mnt/log/hadoop/hadoop//hadoop-hadoop-namenode-cluster-1-master-001.out
> /usr/sbin/hadoop-daemon.sh (http://hadoop-daemon.sh): line 136:
> /mnt/log/hadoop/hadoop//hadoop-hadoop-namenode-cluster-1-master-001.out: No
> such file or directory
> head: cannot open
> `/mnt/log/hadoop/hadoop//hadoop-hadoop-namenode-cluster-1-master-001.out' for
> reading: No such file or directory
> hadoop@localhost's password:
>
> when I try to start hadoop manually, it asks for a password. How to do it? Am
> I missing something?
>
> Regards,
> Arindam
>
>
> --
> Mailing list: https://launchpad.net/~savanna-all
> Post to : savanna-...@lists.launchpad.net
> (mailto:savanna-...@lists.launchpad.net)
> Unsubscribe : https://launchpad.net/~savanna-all
> More help : https://help.launchpad.net/ListHelp
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [savanna]error while accessing Savanna UI
Congratulations!
On Jul 15, 2013, at 6:13 PM, Arindam Choudhury wrote:
> Its solved.
>
> I started a cluster and then its working.
>
>> Date: Mon, 15 Jul 2013 09:57:20 -0400
>> From: m...@redhat.com
>> To: openstack-dev@lists.openstack.org
>> CC: arin...@live.com; savanna-...@lists.launchpad.net
>> Subject: Re: [openstack-dev] [savanna]error while accessing Savanna UI
>>
>> On 07/15/2013 08:45 AM, Arindam Choudhury wrote:
>>> Hi,
>>>
>>> I did:
>>>
>>> git clone https://github.com/stackforge/savanna-dashboard.git
>>>
>>> cd savanna-dashboard
>>>
>>> python setup.py install
>>>
>>> pip show savannadashboard
>>> ---
>>> Name: savannadashboard
>>> Version: 0.2.rc2
>>> Location:
>>> /usr/lib/python2.6/site-packages/savannadashboard-0.2.rc2-py2.6.egg
>>> Requires:
>>>
>>> then in /usr/share/openstack-dashboard/openstack_dashboard/settings.py
>>>
>>> HORIZON_CONFIG = {
>>> 'dashboards': ('project', 'admin', 'settings', 'savanna',),
>>>
>>>
>>> INSTALLED_APPS = (
>>> 'openstack_dashboard',
>>> 'savannadashboard',
>>>
>>> and in
>>> /usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.py
>>>
>>> SAVANNA_URL = 'http://localhost:8386/v1.0'
>>>
>>> But whenever I try to access savanna dashboard I get the following error
>>> in httpd error_access log:
>>>
>>> [Mon Jul 15 07:44:35 2013] [error] ERROR:django.request:Internal Server
>>> Error: /dashboard/savanna/
>>> [Mon Jul 15 07:44:35 2013] [error] Traceback (most recent call last):
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/django/core/handlers/base.py", line
>>> 111, in get_response
>>> [Mon Jul 15 07:44:35 2013] [error] response = callback(request,
>>> *callback_args, **callback_kwargs)
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/horizon/decorators.py", line 38, in dec
>>> [Mon Jul 15 07:44:35 2013] [error] return view_func(request, *args,
>>> **kwargs)
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/horizon/decorators.py", line 54, in dec
>>> [Mon Jul 15 07:44:35 2013] [error] return view_func(request, *args,
>>> **kwargs)
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/horizon/decorators.py", line 38, in dec
>>> [Mon Jul 15 07:44:35 2013] [error] return view_func(request, *args,
>>> **kwargs)
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/django/views/generic/base.py", line
>>> 48, in view
>>> [Mon Jul 15 07:44:35 2013] [error] return self.dispatch(request,
>>> *args, **kwargs)
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/django/views/generic/base.py", line
>>> 69, in dispatch
>>> [Mon Jul 15 07:44:35 2013] [error] return handler(request, *args,
>>> **kwargs)
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/horizon/tables/views.py", line 155, in get
>>> [Mon Jul 15 07:44:35 2013] [error] handled = self.construct_tables()
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/horizon/tables/views.py", line 146, in
>>> construct_tables
>>> [Mon Jul 15 07:44:35 2013] [error] handled = self.handle_table(table)
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/horizon/tables/views.py", line 118, in
>>> handle_table
>>> [Mon Jul 15 07:44:35 2013] [error] data = self._get_data_dict()
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/horizon/tables/views.py", line 182, in
>>> _get_data_dict
>>> [Mon Jul 15 07:44:35 2013] [error] self._data =
>>> {self.table_class._meta.name: self.get_data()}
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/savannadashboard-0.2.rc2-py2.6.egg/savannadashboard/clusters/views.py",
>>> line 40, in get_data
>>> [Mon Jul 15 07:44:35 2013] [error] clusters = savanna.clusters.list()
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/savannadashboard-0.2.rc2-py2.6.egg/savannadashboard/api/clusters.py",
>>> line 74, in list
>>> [Mon Jul 15 07:44:35 2013] [error] return self._list('/clusters',
>>> 'clusters')
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/savannadashboard-0.2.rc2-py2.6.egg/savannadashboard/api/base.py",
>>> line 84, in _list
>>> [Mon Jul 15 07:44:35 2013] [error] resp = self.api.client.get(url)
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/savannadashboard-0.2.rc2-py2.6.egg/savannadashboard/api/httpclient.py",
>>> line 28, in get
>>> [Mon Jul 15 07:44:35 2013] [error] headers={'x-auth-token': self.token})
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/requests/api.py", line 55, in get
>>> [Mon Jul 15 07:44:35 2013] [error] return request('get', url, **kwargs)
>>> [Mon Jul 15 07:44:35 2013] [error] File
>>> "/usr/lib/python2.6/site-packages/requests/api.py", line 44, in request
>>> [Mon Jul 15
Re: [openstack-dev] savanna version 0.3 - added UI mockups for EDP workflow
Chad, I'd like to see more details about job progress on the "Job list view". It should display current progress, logs, errors. For Hive, Pig and Oozie flows it would be useful to list all the jobs from the task. Something similar to https://github.com/twitter/ambrose would be great (without fancy graphs). Ruslan On Fri, Jul 12, 2013 at 7:14 PM, Chad Roberts wrote: > I have added some initial UI mockups for version 0.3. > Any comments are appreciated. > > https://wiki.openstack.org/wiki/Savanna/UIMockups/JobCreation > > Thanks, > Chad > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Savanna] Savanna 0.3 features
I'd like to start conversation on the subject of EDP(Elastic Data Processing) features we are going to implement in Savanna 0.3. Here is a link to etherpad: https://etherpad.openstack.org/edp_v3_components Regards, Ruslan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [savanna] scalable architecture
This question was asked several times. So we decided to provide a detailed response. 1. The first question is “Why doesn’t Savanna use Heat to provision VMs?” Generally using Heat underneath for infrastructure provisioning looks reasonable. In a tactic perspective there are few factors making Heat usage underneath Savanna problematic: * Heat stability for Grizzly release. Savanna currently maintains Grizzly+ compatibility. * Installation of large Hadoop clusters (100+ nodes). Will be addressed by proposed architecture changes. * Anti-affinity support for HDFS redundancy in cloud environment * Circular dependencies - we should generate ‘/etc/hosts’ for all instances in provisioned cluster. We can’t use cloud init for this directly. There are a couple possible solutions using Heat, but none of them looks like a straightforward solution. * Level of complexity. We try to keep things as simple as possible. Adding extra layer will increase overall complexity of the solution. In addition both Savanna and Heat under active development changing lots of internals and even APIs and will require extra effort to coordinate. Here is what we’ll do: * Create a wiki page with text from this email * Create a list of requirements for Heat Once Heat fulfills all the requirements we will be able and should use Heat for VM provisioning. 2. Let’s answer the second question - why we need Savanna? Can’t we use Heat to do what Savanna does? * Savanna provides bunch of Hadoop-specific features. It’ll be hard to provide them as Heat plugin * Savanna provides Hadoop-specific APIs and functionality. Heat use cases are mostly around provisioning/deployment. * Savanna provides integration with various Hadoop distributions through pluggable mechanism Now, more details on each item. Hadoop specific features: * Tight Swift integration. Hadoop can read and write from/to Swift object storage. Savanna provides required configs for the Hadoop cluster. * Usage of anti-affinity to preserve data-redundancy of HDFS nodes Hadoop-specific APIs and functionality: * Hadoop cluster scaling * Elastic Data Processing: https://wiki.openstack.org/wiki/Savanna/EDP Integration with Hadoop distributions through pluggable mechanism: - Usually Hadoop cluster deployment is a multi-step operation. First step is to install management console (for instance Apache Ambari). Second step is to communicate with management console through REST API to provision Hadoop on the cluster. Savanna wraps all this operations under well-defined API. I hope all the items above explain why we need Savanna as a separate OpenStack service. 3. Why can’t Savanna be used as a plugin for Heat? It should be and it will be someday. Regards, Ruslan On Thu, Jul 25, 2013 at 7:42 PM, Joe Gordon wrote: > > On Jul 23, 2013 12:34 PM, "Sergey Lukjanov" wrote: >> >> Hi evereyone, >> >> We’ve started working on upgrading Savanna architecture in version 0.3 to >> make it horizontally scalable. >> >> The most part of information is in the wiki page - >> https://wiki.openstack.org/wiki/Savanna/NextGenArchitecture. >> >> Additionally there are several blueprints created for this activity - >> https://blueprints.launchpad.net/savanna?searchtext=ng- >> >> We are looking for comments / questions / suggestions. > > This sounds like most of this can be built around Heat, except maybe the > rest api to hadoop. So why not use heat for the deploy part? > >> >> P.S. The another thing that we’re working on in Savanna 0.3 is EDP >> (Elastic Data Processing). >> >> Thank you! >> >> Sincerely yours, >> Sergey Lukjanov >> Savanna Technical Lead >> Mirantis Inc. >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [savanna] savanna-extra and hadoop-patch
Matt, >From the bug description: Affects Version/s:1.2.0, 2.0.3-alpha Target Version/s: 3.0.0, 2.3.0 So, it seems that Hadoop folks don't intend to include this patch into Hadoop 1.x Ruslan On Tuesday, August 27, 2013 at 2:41 PM, Matthew Farrellee wrote: > Howdy Ivan, > > FYI, https://issues.apache.org/jira/browse/HADOOP-8545 is currently > targeting 1.2.0 and 2.0.3-alpha. And the code (HADOOP-8545-034.patch) > appears to provide support to Hadoop 1.x HDFS, though I may be missing > something. > > I'd suggest only adding a Swift HCFS repo only if the code is not > destined to go to Apache Hadoop. > > +1 discuss at meeting > > Best, > > > s/matt/erik/ > > On 08/27/2013 01:45 AM, Sergey Lukjanov wrote: > > Hi Erik, > > > > First of all, savanna-extra has been created exactly for such needs - > > to store all stuff that we need but couldn't be placed to another > > repos. Initially it contains elements and pre-builded jar with Swift > > HCFS. Now the last one has been moved to the CDN and it's a good idea > > to make separated project for elements. > > > > As for Swift HCFS the core attached to the HADOOP-8545 is targeted to > > the Hadoop 2 version and should be patched to work with Hadoop 1.X > > correctly. So, that's why we add it to the extra repo. It looks like > > that it's ok to add one more repo for Swift HCFS near the savanna at > > stackforge like HCFS for Gluster[0]. > > > > So, let's discuss both of the migrations at the next IRC team meeting. > > > > [0] https://github.com/gluster/hadoop-glusterfs > > > > Sincerely yours, > > Sergey Lukjanov > > Savanna Technical Lead > > Mirantis Inc. > > > > On Aug 27, 2013, at 5:18, Matthew Farrellee > (mailto:m...@redhat.com)> wrote: > > > > > https://review.openstack.org/#/c/42926/ > > > > > > I didn't get back to this on Friday and it got merged this morning, so > > > here's my feedback. > > > > > > The savanna-extra repository now appears to hold (a) DIB image elements > > > as well as (b) the source for the Swift backed HCFS (Hadoop Compatible > > > File System) implementation. > > > > > > If I understand this correctly, (b) is actually the patch set that is > > > being proposed to the Apache Hadoop community. That patch set has not > > > been accepted and is being tracked in HADOOP-8545[0], which appears > > > stalled since July 2013. > > > > > > Let's break Savanna's DIB elements out of savanna-extra and into > > > savanna-image-elements. It has a clear path forward and a good definition > > > of scope. > > > > > > Let's also leave savanna-extra as a grab bag, whose only occupant is > > > currently the Swift code. Eventually that code will need a proper home, > > > either contributed to Apache Hadoop or broken out as its own project. > > > > > > Best, > > > > > > > > > matt > > > > > > [0] https://issues.apache.org/jira/browse/HADOOP-8545 > > > > > > ___ > > > OpenStack-dev mailing list > > > OpenStack-dev@lists.openstack.org > > > (mailto:OpenStack-dev@lists.openstack.org) > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org (mailto:OpenStack-dev@lists.openstack.org) > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org (mailto:OpenStack-dev@lists.openstack.org) > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [fuel] Branching strategy vs feature freeze
On Wed, Aug 26, 2015 at 4:23 AM, Igor Marnat wrote: > Thierry, Dmitry, > key point is that we in Fuel need to follow as much community adopted > process as we can, and not to introduce something Fuel specific. We > need not only to avoid forking code, but also to avoid forking > processes and approaches for Fuel. > > Both #2 and #3 allow it, making it easier for contributors to > participate in the Fuel. > > #3 (having internal repos for pre-release preparation, bug fixing and > small custom features) from community perspective is the same as #2, > provided that all the code from these internal repos always ends up > being committed upstream. Purely internal logistics. > > The only one additional note from my side is that we might want to > consider an approach slightly adopted similar to what's there in > Puppet modules. AFAIK, they are typically released several weeks later > than Openstack code. This is natural for Fuel as it depends on these > modules and packaging of Openstack. > I also think we should go with option #2. What it means to me * Short FF: create stable branch couple of weeks after FF for upstream Fuel * Untie release schedule for upstream Fuel and MOS. This should be two separate schedules * Fuel release schedule should be more aligned with OpenStack release schedule. It should be similar to upstream OpenStack Puppet schedule, where Puppet modules are developed during the same time frame as OpenStack and released just a few weeks later * Internally vendors can have downstream branches (which is option #3 from Dmitry’s email) Thanks, Ruslan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Congress] [Murano] Congress devstack installation is failing in murano gate job
On Wed, Apr 29, 2015 at 6:46 PM, Davanum Srinivas wrote: > So, summarizing some chatter on #openstack-dev: > > Action item for unblocking the gate job: > 1. Remove the -U in the shell script that installs > thirdparty-requirements.txt in congress' repo. This patch to Congress stable/kilo resolved the issue with failing job gate-murano-congress-devstack-dsvm: https://review.openstack.org/#/c/178777/ > If that's not enough: > 2. make sure congress' stable/kilo > requirements.txt/test-requirements.txt is sync'ed with the global > stable/kilo. > > -- dims __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
