[openstack-dev] [fuel] Weekly meeting Jan 19 is canceled
Nothing is on the agenda [0] this week, so I'm calling to cancel the meeting. If you have anything to discuss please come chat in #fuel or add it to the agenda to discuss next week. [0] https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda --- WBR, Alexey Shtokolov OpenStack Fuel PTL __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [fuel] Weekly meetings Dec22, Dec29 and Jan5 are cancelled
Fuelers, As we agreed next 3 meetings (Dec22, Dec29 and Jan5) are cancelled due to holidays. Please feel free to put your topics to agenda [0] for Jan12 meeting. [0] https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda --- WBR, Alexey Shtokolov OpenStack Fuel PTL __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [fuel] Weekly meeting Nov 10 is canceled
Nothing is on the agenda [0] this week, so I'm calling to cancel the meeting. If you have anything to discuss. Please come chat in #fuel or add it to the agenda to discuss next week. [0] https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda --- WBR, Alexey Shtokolov OpenStack Fuel PTL __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] Weekly meetings for 10/20 and 10/27 are canceled
Fuelers, Due to preparation to the upcoming summit and our summit sessions [0] We decided to cancel this week and next week's Fuel IRC meetings. [0] - https://etherpad.openstack.org/p/fuel-ocata-summit-planning --- WBR, Alexey Shtokolov Fuel PTL __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][ansible][fuel][kolla][puppet][tripleo] proposed deadlines for cycle-trailing projects
Doug, We've finally fixed the blockers and tagged RC1 for all our repos. We're going to have final RC this Friday (Oct14). Could I ask you to create stable/newton branches for repos: - openstack/fuel-agent - openstack/fuel-astute - openstack/fuel-library - openstack/fuel-main - openstack/fuel-menu - openstack/fuel-nailgun-agent - openstack/fuel-ostf - openstack/fuel-qa - openstack/fuel-ui - openstack/fuel-virtualbox - openstack/fuel-web based on the tag 10.0.0rc1 Or should I do it myself? Best regards, Alexey Shtokolov On Fri, Oct 7, 2016 at 10:16 PM, Doug Hellmann <d...@doughellmann.com> wrote: > This week we tagged the final releases for projects using the > cycle-with-milestones release model. Projects using the cycle-trailing > model have two more weeks before their final release tags are due. In > the time between now and then, we expect those projects to be preparing > and tagging release candidates. > > Just as with the milestone-based projects, we want to manage the number, > frequency, and timing of release candidates for cycle-trailing projects. > With that in mind, I would like to propose the following rough timeline > (my apologies for not preparing this sooner): > > 10 Oct -- All cycle-trailing projects tag at least their first RC. > 13 Oct -- Soft deadline for cycle-trailing projects to tag a final RC. > 18 Oct -- Hard deadline for cycle-trailing projects to tag a final RC. > 20 Oct -- Re-tag the final RCs as a final release. > > Between the first and later release candidates, any translations and > bug fixes should be merged. > > We want to leave a few days between the last release candidate and > the final release so that downstream consumers of the projects can > report issues against stable artifacts. Given the nature of most > of our trailing projects, and the lateness of starting to discuss > these deadlines, I don't think we need the same amount of time as > we usually set aside for the milestone-based projects. Based on > that assumption, I've proposed a 1 week soft goal and a 2 day hard > deadline. > > Let me know what you think, > Doug > > Newton schedule: https://releases.openstack.org/newton/schedule.html > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] [ptl] Announcing my candidacy for Fuel PTL in the Ocala cycle
Fuelers, I would like to announce my candidacy for Fuel PTL in the Ocata cycle. ** Introduction ** First of all, allow me to introduce myself. I joined the OpenStack Community during the Juno release cycle. I started on a "consumer" side - as a Head of IT at a media holding I was consuming and bringing Open Source technologies to an "Enterprise world" as much as it was possible and OpenStack was there as well. I had been investigating OpenStack and finally deployed my first private cloud manually. It was Juno 2014.2 and it was a very-very long trip. Half a year later I found out that Fuel could do it much more faster! I liked Fuel mission, both stated and delivered - make OpenStack installation process easier and more streamlined. OpenStack was in a serious need of such a tool, as one of prerequisites for lowering adoption barriers. Finally, I joined Fuel Community as a deployment engineer. Since 2015 I've been an active contributor in the very core features of Fuel and have been leading a Fuel development team [0] at Mirantis. If you're using or developing Fuel - you’ve probably heard of some of the features that we designed and implemented: Task-based deployment engine [1], Post-deployment plugins installation, Deployment History [2][3], Data-driven orchestration [4] with Unlocked Settings and Network tabs [5] (aka Fuel LCM), Custom graphs execution [6], Release-as-a-Plugin [7] and Everything-as-a-Graph [8]. As you can see, the main goal of this work was to make Fuel more flexible and friendly for deployment engineers, based on my and my colleagues’ experience from real OpenStack deployments. And I would like to keep doing it as a Fuel PTL. ** Community ** Fuel, as a part of OpenStack BigTent since November 17 2015, had the first design summit at Austin. Vladimir Kozhukalov did a great job organizing it. This summit allowed Fuel community to solicit a feedback, to align our goals with OpenStack community and, the most important, to start having an open design. Open Design was the last of four Opens [9] to be achieved by Fuel Community. Unfortunately during the Newton release cycle not all discussions and decisions were open and public enough. We need to avoid such behaviour in future. Fuel Community will have a very modest representation during upcoming Fuel Ocata Design summit at Barcelona and the first Project Teams Gathering [10] And that’s the reason to make our goal #1: the expansion of the OpenStack Community involvement into Fuel design during the whole release cycle. And as a result encourage new contributors, build a healthy community around Fuel and achieve the contribution diversity. ** Technical challenges ** October 2015 Fuel was the 5th deployment tool for OpenStack with 12% [11] but April 2016 Fuel got the 3rd place with 19% [12] right after Puppet and Ansible. While SaltStack, Chef and Puppet are visibly going down Fuel is taking over the OpenStack deployment world. But we still have a set of technical challenges and we must focus on: * Infrastructure-as-Code (IaC) allows us to provide convenient and familiar UX for those user who have experience with dynamic infrastructure in Public clouds - it's now easy to control OpenStack configuration and versioning via git. Since nailgun extension for IaC [13] became a part of Fuel project [14] we should actively support and develop it. * Lifecycle management. Making changes on already installed clouds is still complicated. We’ve done a lot of work in this direction, but we are still not there. * The Upgrades of Fuel itself and OpenStack clusters managed by Fuel still need more flexibility and much better UX. We can fix it using the custom graphs concept [8]. * Fuel extensions and plugins framework should be polished and provide itself with stable API. Moreover extensions and plugins SDK should become developer-friendly and support all new features like release-as-a-plugin [7]. * Single Fuel Master node should support multiple different OpenStack releases. First step [7] was done. Let’s keep moving. * Audit, troubleshooting and debugging became easier with Deployment History features and dry-run/noop deployments but we are still facing with the inconsistent error reporting and diagnostic snapshots. We can improve it using graphs. * Documentation team has done a great job of restructuring Fuel documentation and moving it under OpenStack Doc space. However, as subject matter experts, we need to proactively review and help the doc team with updates and new features documentation [15] ** Afterword ** As an epilogue I would like to say: Working closely with OpenStack Community I see that PTL is more about how to help people come together and achieve goals, how to built horizontal communications between project teams and how to keep developers in their zone of maximum performance. Fuel is a mature project with existing strong development team. And we should keep these things in place. WBR, Alexey Shtokolov
Re: [openstack-dev] [fuel-library] Nominate Stanislaw Bogatkin to fuel-library core
+1 Stanislaw made a great contribution! I would like to mention SSL-support, YAQL expressions for data-driven orchestration and his awesome live YAQL evaluator for Fuel Master node [0] [0] https://github.com/sorrowless/fuyaql --- WBR, Alexey Shtokolov Fuel Development Lead On Thu, Sep 8, 2016 at 1:17 PM, Denis Egorenko <degore...@mirantis.com> wrote: > +1 > > 2016-09-08 13:06 GMT+03:00 Aleksandr Didenko <adide...@mirantis.com>: > >> +1 >> >> On Thu, Sep 8, 2016 at 11:06 AM, Sergey Vasilenko < >> svasile...@mirantis.com> wrote: >> >>> +1 >>> >>> >>> >>> >>> __ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: openstack-dev-requ...@lists.op >>> enstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > -- > Best Regards, > Egorenko Denis, > Senior Deployment Engineer > Mirantis > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Nominate Ilya Kutukov for fuel-plugins-core
My strong +1 Ilya made and keeps making a great contribution to the development of Fuel Plugins Framework on both sides (nailgun and plugins) especially during the Newton release cycle! --- WBR, Alexey Shtokolov Fuel Development Lead On Sun, Sep 11, 2016 at 5:41 PM, Aleksey Kasatkin <akasat...@mirantis.com> wrote: > +1 > > > Aleksey Kasatkin > > > On Sat, Sep 10, 2016 at 4:49 PM, Sergey Vasilenko <svasile...@mirantis.com > > wrote: > >> +1 >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How to run a task when multiple tasks are completed
Hi! In case when all 4 tasks are on the same node you should use 'requires' and 'required_for' fields in the task definition to make dependencies between them [0]. E.g.: *- id: Task4* *version: [a version of the tasks graph execution engine] * * type: [one of: stage, group, skipped, puppet, shell] * * role: [matches roles for which this tasks should be executed] * *requires: [Task1, Task2, Task3]* In case of cross-node dependencies you can use 'cross-depends' and 'cross-depended-by' [0]. [0] - https://docs.mirantis.com/openstack/fuel/fuel-8.0/reference-architecture.html#task-based-deployment 2016-05-25 14:16 GMT+03:00 Jonnalagadda, Venkata < venkata.jonnalaga...@intl.att.com>: > Fuel Team, > > > > I have couple of tasks in Fuel (deployment_tasks.yaml) as below – > > > > Task1 > > Task2 > > Task3 > > Task4 > > > > Now, I want to run Task4 only when Tasks-1,2,3 are completed. How I can > configure this in deployment_tasks yaml ? Please suggest. > > > > *Thanks & Regards,* > > > > *J. Venkata Mahesh* > > > > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- --- WBR, Alexey Shtokolov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Newton Design Summit sessions planning
Hi, +1 from my side. --- WBR, Alexey Shtokolov 2016-04-14 16:47 GMT+03:00 Evgeniy L <e...@mirantis.com>: > Hi, no problem from my side. > > On Thu, Apr 14, 2016 at 10:53 AM, Vladimir Kozhukalov < > vkozhuka...@mirantis.com> wrote: > >> Dear colleagues, >> >> I'd like to request workrooms sessions swap. >> >> We have a session about Fuel/Ironic integration and I'd like >> this session not to overlap with Ironic sessions, so Ironic >> team could attend Fuel sessions. At the same time, we have >> a session about orchestration engine and it would be great to >> invite there people from Mistral and Heat. >> >> My suggestion is as follows: >> >> Wed: >> 9:50 Astute -> Mistral/Heat/??? >> Thu: >> 9.00 Fuel/Ironic/Ironic-inspector >> >> If there are any objections, please let me know asap. >> >> >> >> Vladimir Kozhukalov >> >> On Fri, Apr 1, 2016 at 9:47 PM, Vladimir Kozhukalov < >> vkozhuka...@mirantis.com> wrote: >> >>> Dear colleagues, >>> >>> Looks like we have final version sessions layout [1] >>> for Austin design summit. We have 3 fishbows, >>> 11 workrooms, full day meetup. >>> >>> Here you can find some useful information about design >>> summit [2]. All session leads must read this page, >>> be prepared for their sessions (agenda, slides if needed, >>> etherpads for collaborative work, etc.) and follow >>> the recommendations given in "At the Design Summit" section. >>> >>> Here is Fuel session planning etherpad [3]. Almost all suggested >>> topics have been put there. Please put links to slide decks >>> and etherpads next to respective sessions. Here is the >>> page [4] where other teams publish their planning pads. >>> >>> If session leads want for some reason to swap their slots it must >>> be requested in this ML thread. If for some reason session lead >>> can not lead his/her session, it must be announced in this ML thread. >>> >>> Fuel sessions are: >>> === >>> Fishbowls: >>> === >>> Wed: >>> 15:30-16:10 >>> 16:30:17:10 >>> 17:20-18:00 >>> >>> === >>> Workrooms: >>> === >>> Wed: >>> 9:00-9:40 >>> 9:50-10:30 >>> 11:00-11:40 >>> 11:50-12:30 >>> 13:50-14:30 >>> 14:40-15:20 >>> Thu: >>> 9:00-9:40 >>> 9:50-10:30 >>> 11:00-11:40 >>> 11:50-12:30 >>> 13:30-14:10 >>> >>> === >>> Meetup: >>> === >>> Fri: >>> 9:00-12:30 >>> 14:00-17:30 >>> >>> [1] >>> http://lists.openstack.org/pipermail/openstack-dev/attachments/20160331/d59d38b7/attachment.pdf >>> [2] https://wiki.openstack.org/wiki/Design_Summit >>> [3] https://etherpad.openstack.org/p/fuel-newton-summit-planning >>> [4] https://wiki.openstack.org/wiki/Design_Summit/Planning >>> >>> Thanks. >>> >>> Vladimir Kozhukalov >>> >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] (no subject)
Fuelers! I'm glad to announce that all patches [0] were merged! Many thanks to all of you who help us to make Fuel more flexible and unlimited. Special thanks to our code and design reviewers: Igor Kalnitsky, Sergii Golovatiuk, Vitaly Kramskikh. And especially to the Team: Bulat Gaifullin, Ilya Kutukov, Vladimir Sharshov, Julia Aranovich, Stanislaw Bogatkin, Evgeniy L and Vladimir Kuklin My sincerest thanks and appreciation for your great efforts, sleepless nights and sense of purpose! WBR, Alexey Shtokolov [0] - https://goo.gl/kSwej5 2016-03-25 21:48 GMT+03:00 Vladimir Kozhukalov <vkozhuka...@mirantis.com>: > Granted. New deadline is 21:00 UTC 03/28/2016. > > Vladimir Kozhukalov > > On Fri, Mar 25, 2016 at 8:17 PM, Alexey Shtokolov <ashtoko...@mirantis.com > > wrote: > >> Fuelers! >> >> >> We are very close to landing our feature "Unlock Settings Tab". But we >> still have a set of reviews [0] to be merged due to several reasons (incl. >> the migration to python27-db gates on OpenStack Infra). I would like to >> request extra time till Monday to land them. >> >> >> [0] - https://goo.gl/kSwej5 >> >> 2016-03-14 11:00 GMT+03:00 Dmitry Borodaenko <dborodae...@mirantis.com>: >> >>> Thanks for working this out! Confirming that task history remains >>> included in the scope of this FFE until the merge deadline, March 24. >>> >>> On Fri, Mar 11, 2016 at 11:48:51PM +0200, Igor Kalnitsky wrote: >>> > Hey Dmitry, >>> > >>> > I confirm that we agreed on feature design, and you can proceed with >>> > granting exception. >>> > >>> > ,- Igor >>> > >>> > On Fri, Mar 11, 2016 at 8:27 PM, Alexey Shtokolov >>> > <ashtoko...@mirantis.com> wrote: >>> > > Hi Dmitry, >>> > > >>> > > We've reached the design consensus with Igor today. Could you please >>> remove >>> > > the conditional status of the FFE request? >>> > > As agreed: the merge deadline is March 24. >>> > > >>> > > -- >>> > > WBR, Alexey Shtokolov >>> > > >>> > > 2016-03-11 2:27 GMT+03:00 Dmitry Borodaenko < >>> dborodae...@mirantis.com>: >>> > >> >>> > >> Granted. Design consensus deadline for the task history part of this >>> > >> feature is extended to March 11. This does not change the merge >>> deadline >>> > >> for other parts of this feature, which is still March 24. >>> > >> >>> > >> -- >>> > >> Dmitry Borodaenko >>> > >> >>> > >> >>> > >> On Fri, Mar 11, 2016 at 01:02:52AM +0300, Alexey Shtokolov wrote: >>> > >> > Dmitry, >>> > >> > >>> > >> > We are really close to have the consensus, but we need one more >>> meeting >>> > >> > with Fuel-Python Component Lead Igor Kalnitsky to make the final >>> > >> > decision. >>> > >> > All patches [0] are on review. The meeting is scheduled for >>> tomorrow >>> > >> > (03/11 >>> > >> > 1:30pm CET). >>> > >> > Could you please grant us one more day for it? >>> > >> > >>> > >> > [0] - >>> > >> > >>> https://review.openstack.org/#/q/topic:bp/store-deployment-tasks-history >>> > >> > >>> > >> > -- >>> > >> > WBR, Alexey Shtokolov >>> > >> > >>> > >> > 2016-03-04 3:13 GMT+03:00 Dmitry Borodaenko < >>> dborodae...@mirantis.com>: >>> > >> > >>> > >> > > Granted, merge deadline March 24, task history part of the >>> feature is >>> > >> > > to >>> > >> > > be excluded from this exception grant unless a consensus is >>> reached by >>> > >> > > March 10. >>> > >> > > >>> > >> > > Relevant part of the meeting log starts at: >>> > >> > > >>> > >> > > >>> > >> > > >>> http://eavesdrop.openstack.org/meetings/fuel/2016/fuel.2016-03-03-16.00.log.html#l-198 >>> > >> > > >>> > >> > > -- >>> > >> > > Dmitry Borodaenko >>> > >> > > >>> >
[openstack-dev] [Fuel] [FFE] Unlock Settings Tab
Fuelers! I'm glad to announce that all patches [0] were merged! Many thanks to all of you who help us to make Fuel more flexible and unlimited. Special thanks to our code and design reviewers: Igor Kalnitsky, Sergii Golovatiuk, Vitaly Kramskikh. And especially to the Team: Bulat Gaifullin, Ilya Kutukov, Vladimir Sharshov, Julia Aranovich, Stanislaw Bogatkin, Evgeniy L and Vladimir Kuklin My sincerest thanks and appreciation for your great efforts, sleepless nights and sense of purpose! WBR, Alexey Shtokolov [0] - https://goo.gl/kSwej5 2016-03-25 21:48 GMT+03:00 Vladimir Kozhukalov <vkozhuka...@mirantis.com>: > Granted. New deadline is 21:00 UTC 03/28/2016. > > Vladimir Kozhukalov > > On Fri, Mar 25, 2016 at 8:17 PM, Alexey Shtokolov <ashtoko...@mirantis.com > > wrote: > >> Fuelers! >> >> >> We are very close to landing our feature "Unlock Settings Tab". But we >> still have a set of reviews [0] to be merged due to several reasons (incl. >> the migration to python27-db gates on OpenStack Infra). I would like to >> request extra time till Monday to land them. >> >> >> [0] - https://goo.gl/kSwej5 >> >> 2016-03-14 11:00 GMT+03:00 Dmitry Borodaenko <dborodae...@mirantis.com>: >> >>> Thanks for working this out! Confirming that task history remains >>> included in the scope of this FFE until the merge deadline, March 24. >>> >>> On Fri, Mar 11, 2016 at 11:48:51PM +0200, Igor Kalnitsky wrote: >>> > Hey Dmitry, >>> > >>> > I confirm that we agreed on feature design, and you can proceed with >>> > granting exception. >>> > >>> > ,- Igor >>> > >>> > On Fri, Mar 11, 2016 at 8:27 PM, Alexey Shtokolov >>> > <ashtoko...@mirantis.com> wrote: >>> > > Hi Dmitry, >>> > > >>> > > We've reached the design consensus with Igor today. Could you please >>> remove >>> > > the conditional status of the FFE request? >>> > > As agreed: the merge deadline is March 24. >>> > > >>> > > -- >>> > > WBR, Alexey Shtokolov >>> > > >>> > > 2016-03-11 2:27 GMT+03:00 Dmitry Borodaenko < >>> dborodae...@mirantis.com>: >>> > >> >>> > >> Granted. Design consensus deadline for the task history part of this >>> > >> feature is extended to March 11. This does not change the merge >>> deadline >>> > >> for other parts of this feature, which is still March 24. >>> > >> >>> > >> -- >>> > >> Dmitry Borodaenko >>> > >> >>> > >> >>> > >> On Fri, Mar 11, 2016 at 01:02:52AM +0300, Alexey Shtokolov wrote: >>> > >> > Dmitry, >>> > >> > >>> > >> > We are really close to have the consensus, but we need one more >>> meeting >>> > >> > with Fuel-Python Component Lead Igor Kalnitsky to make the final >>> > >> > decision. >>> > >> > All patches [0] are on review. The meeting is scheduled for >>> tomorrow >>> > >> > (03/11 >>> > >> > 1:30pm CET). >>> > >> > Could you please grant us one more day for it? >>> > >> > >>> > >> > [0] - >>> > >> > >>> https://review.openstack.org/#/q/topic:bp/store-deployment-tasks-history >>> > >> > >>> > >> > -- >>> > >> > WBR, Alexey Shtokolov >>> > >> > >>> > >> > 2016-03-04 3:13 GMT+03:00 Dmitry Borodaenko < >>> dborodae...@mirantis.com>: >>> > >> > >>> > >> > > Granted, merge deadline March 24, task history part of the >>> feature is >>> > >> > > to >>> > >> > > be excluded from this exception grant unless a consensus is >>> reached by >>> > >> > > March 10. >>> > >> > > >>> > >> > > Relevant part of the meeting log starts at: >>> > >> > > >>> > >> > > >>> > >> > > >>> http://eavesdrop.openstack.org/meetings/fuel/2016/fuel.2016-03-03-16.00.log.html#l-198 >>> > >> > > >>> > >> > > -- >>> > >> > > Dmitry Borodaenko >>> > >> > > >>> >
Re: [openstack-dev] [Fuel] [FFE] Unlock Settings Tab
Fuelers! We are very close to landing our feature "Unlock Settings Tab". But we still have a set of reviews [0] to be merged due to several reasons (incl. the migration to python27-db gates on OpenStack Infra). I would like to request extra time till Monday to land them. [0] - https://goo.gl/kSwej5 2016-03-14 11:00 GMT+03:00 Dmitry Borodaenko <dborodae...@mirantis.com>: > Thanks for working this out! Confirming that task history remains > included in the scope of this FFE until the merge deadline, March 24. > > On Fri, Mar 11, 2016 at 11:48:51PM +0200, Igor Kalnitsky wrote: > > Hey Dmitry, > > > > I confirm that we agreed on feature design, and you can proceed with > > granting exception. > > > > ,- Igor > > > > On Fri, Mar 11, 2016 at 8:27 PM, Alexey Shtokolov > > <ashtoko...@mirantis.com> wrote: > > > Hi Dmitry, > > > > > > We've reached the design consensus with Igor today. Could you please > remove > > > the conditional status of the FFE request? > > > As agreed: the merge deadline is March 24. > > > > > > -- > > > WBR, Alexey Shtokolov > > > > > > 2016-03-11 2:27 GMT+03:00 Dmitry Borodaenko <dborodae...@mirantis.com > >: > > >> > > >> Granted. Design consensus deadline for the task history part of this > > >> feature is extended to March 11. This does not change the merge > deadline > > >> for other parts of this feature, which is still March 24. > > >> > > >> -- > > >> Dmitry Borodaenko > > >> > > >> > > >> On Fri, Mar 11, 2016 at 01:02:52AM +0300, Alexey Shtokolov wrote: > > >> > Dmitry, > > >> > > > >> > We are really close to have the consensus, but we need one more > meeting > > >> > with Fuel-Python Component Lead Igor Kalnitsky to make the final > > >> > decision. > > >> > All patches [0] are on review. The meeting is scheduled for tomorrow > > >> > (03/11 > > >> > 1:30pm CET). > > >> > Could you please grant us one more day for it? > > >> > > > >> > [0] - > > >> > > https://review.openstack.org/#/q/topic:bp/store-deployment-tasks-history > > >> > > > >> > -- > > >> > WBR, Alexey Shtokolov > > >> > > > >> > 2016-03-04 3:13 GMT+03:00 Dmitry Borodaenko < > dborodae...@mirantis.com>: > > >> > > > >> > > Granted, merge deadline March 24, task history part of the > feature is > > >> > > to > > >> > > be excluded from this exception grant unless a consensus is > reached by > > >> > > March 10. > > >> > > > > >> > > Relevant part of the meeting log starts at: > > >> > > > > >> > > > > >> > > > http://eavesdrop.openstack.org/meetings/fuel/2016/fuel.2016-03-03-16.00.log.html#l-198 > > >> > > > > >> > > -- > > >> > > Dmitry Borodaenko > > >> > > > > >> > > > > >> > > On Wed, Mar 02, 2016 at 06:00:40PM +0700, Vitaly Kramskikh wrote: > > >> > > > Oh, so there is a spec. I was worried that this patch has > > >> > > > "WIP-no-bprint-assigned-yet" string in the commit message, so I > > >> > > > thought > > >> > > > there is no spec for it. So the commit message should be > updated to > > >> > > > avoid > > >> > > > such confusion. > > >> > > > > > >> > > > It's really good I've seen this spec. There are plans to > overhaul UI > > >> > > > data > > >> > > > format description which we use for cluster and node settings to > > >> > > > solve > > >> > > some > > >> > > > issues and implement long-awaited features like nested > structures, > > >> > > > so we > > >> > > > might also want to deprecate our expression language and also > switch > > >> > > > to > > >> > > > YAQL (and thus port YAQL to JS). > > >> > > > > > >> > > > 2016-03-02 17:17 GMT+07:00 Vladimir Kuklin < > vkuk...@mirantis.com>: > > >> > > > > > >> > > > > Vitaly > > >> > > > > > > >&g
Re: [openstack-dev] [Fuel] [FFE] Unlock Settings Tab
Hi Dmitry, We've reached the design consensus with Igor today. Could you please remove the conditional status of the FFE request? As agreed: the merge deadline is March 24. -- WBR, Alexey Shtokolov 2016-03-11 2:27 GMT+03:00 Dmitry Borodaenko <dborodae...@mirantis.com>: > Granted. Design consensus deadline for the task history part of this > feature is extended to March 11. This does not change the merge deadline > for other parts of this feature, which is still March 24. > > -- > Dmitry Borodaenko > > > On Fri, Mar 11, 2016 at 01:02:52AM +0300, Alexey Shtokolov wrote: > > Dmitry, > > > > We are really close to have the consensus, but we need one more meeting > > with Fuel-Python Component Lead Igor Kalnitsky to make the final > decision. > > All patches [0] are on review. The meeting is scheduled for tomorrow > (03/11 > > 1:30pm CET). > > Could you please grant us one more day for it? > > > > [0] - > > https://review.openstack.org/#/q/topic:bp/store-deployment-tasks-history > > > > -- > > WBR, Alexey Shtokolov > > > > 2016-03-04 3:13 GMT+03:00 Dmitry Borodaenko <dborodae...@mirantis.com>: > > > > > Granted, merge deadline March 24, task history part of the feature is > to > > > be excluded from this exception grant unless a consensus is reached by > > > March 10. > > > > > > Relevant part of the meeting log starts at: > > > > > > > http://eavesdrop.openstack.org/meetings/fuel/2016/fuel.2016-03-03-16.00.log.html#l-198 > > > > > > -- > > > Dmitry Borodaenko > > > > > > > > > On Wed, Mar 02, 2016 at 06:00:40PM +0700, Vitaly Kramskikh wrote: > > > > Oh, so there is a spec. I was worried that this patch has > > > > "WIP-no-bprint-assigned-yet" string in the commit message, so I > thought > > > > there is no spec for it. So the commit message should be updated to > avoid > > > > such confusion. > > > > > > > > It's really good I've seen this spec. There are plans to overhaul UI > data > > > > format description which we use for cluster and node settings to > solve > > > some > > > > issues and implement long-awaited features like nested structures, > so we > > > > might also want to deprecate our expression language and also switch > to > > > > YAQL (and thus port YAQL to JS). > > > > > > > > 2016-03-02 17:17 GMT+07:00 Vladimir Kuklin <vkuk...@mirantis.com>: > > > > > > > > > Vitaly > > > > > > > > > > Thanks for bringing this up. Actually the spec has been on review > for > > > > > almost 2 weeks: https://review.openstack.org/#/c/282695/. > Essentially, > > > > > this is not introducing new DSL but replacing the existing one with > > > more > > > > > powerful extendable language which is being actively developed > within > > > > > OpenStack and is already a part of other projects (Murano, > Mistral), > > > which > > > > > has much more contributors, can return not only boolean but any > > > arbitrary > > > > > collections. So it means that we want to deprecate current > Expression > > > > > language that you wrote and replace it with YAQL due to those > reasons. > > > You > > > > > are not going to extend this Expression-based language in 3 weeks > up to > > > > > level of support of extensions, method overloading, return of > arbitrary > > > > > collections (e.g. we also want to calculate cross-depends and > requires > > > > > fields on the fly which require for it to return list of dicts) and > > > support > > > > > of this stuff on your own, are you? > > > > > > > > > > On Wed, Mar 2, 2016 at 10:09 AM, Vitaly Kramskikh < > > > vkramsk...@mirantis.com > > > > > > wrote: > > > > > > > > > >> I think it's not a part of best practices to introduce changes > like > > > > >> https://review.openstack.org/#/c/279714/ (adding yet another DSL > to > > > the > > > > >> project) without a blueprint and review and discussion of the > spec. > > > > >> > > > > >> 2016-03-02 2:19 GMT+07:00 Alexey Shtokolov < > ashtoko...@mirantis.com>: > > > > >> > > > > >>> Fuelers, > > > > >>> > > > > >>> I would like
Re: [openstack-dev] [Fuel] [FFE] Unlock Settings Tab
Dmitry, We are really close to have the consensus, but we need one more meeting with Fuel-Python Component Lead Igor Kalnitsky to make the final decision. All patches [0] are on review. The meeting is scheduled for tomorrow (03/11 1:30pm CET). Could you please grant us one more day for it? [0] - https://review.openstack.org/#/q/topic:bp/store-deployment-tasks-history -- WBR, Alexey Shtokolov 2016-03-04 3:13 GMT+03:00 Dmitry Borodaenko <dborodae...@mirantis.com>: > Granted, merge deadline March 24, task history part of the feature is to > be excluded from this exception grant unless a consensus is reached by > March 10. > > Relevant part of the meeting log starts at: > > http://eavesdrop.openstack.org/meetings/fuel/2016/fuel.2016-03-03-16.00.log.html#l-198 > > -- > Dmitry Borodaenko > > > On Wed, Mar 02, 2016 at 06:00:40PM +0700, Vitaly Kramskikh wrote: > > Oh, so there is a spec. I was worried that this patch has > > "WIP-no-bprint-assigned-yet" string in the commit message, so I thought > > there is no spec for it. So the commit message should be updated to avoid > > such confusion. > > > > It's really good I've seen this spec. There are plans to overhaul UI data > > format description which we use for cluster and node settings to solve > some > > issues and implement long-awaited features like nested structures, so we > > might also want to deprecate our expression language and also switch to > > YAQL (and thus port YAQL to JS). > > > > 2016-03-02 17:17 GMT+07:00 Vladimir Kuklin <vkuk...@mirantis.com>: > > > > > Vitaly > > > > > > Thanks for bringing this up. Actually the spec has been on review for > > > almost 2 weeks: https://review.openstack.org/#/c/282695/. Essentially, > > > this is not introducing new DSL but replacing the existing one with > more > > > powerful extendable language which is being actively developed within > > > OpenStack and is already a part of other projects (Murano, Mistral), > which > > > has much more contributors, can return not only boolean but any > arbitrary > > > collections. So it means that we want to deprecate current Expression > > > language that you wrote and replace it with YAQL due to those reasons. > You > > > are not going to extend this Expression-based language in 3 weeks up to > > > level of support of extensions, method overloading, return of arbitrary > > > collections (e.g. we also want to calculate cross-depends and requires > > > fields on the fly which require for it to return list of dicts) and > support > > > of this stuff on your own, are you? > > > > > > On Wed, Mar 2, 2016 at 10:09 AM, Vitaly Kramskikh < > vkramsk...@mirantis.com > > > > wrote: > > > > > >> I think it's not a part of best practices to introduce changes like > > >> https://review.openstack.org/#/c/279714/ (adding yet another DSL to > the > > >> project) without a blueprint and review and discussion of the spec. > > >> > > >> 2016-03-02 2:19 GMT+07:00 Alexey Shtokolov <ashtoko...@mirantis.com>: > > >> > > >>> Fuelers, > > >>> > > >>> I would like to request a feature freeze exception for "Unlock > settings > > >>> tab" feature [0] > > >>> > > >>> This feature being combined with Task-based deployment [1] and > > >>> LCM-readiness for Fuel deployment tasks [2] unlocks Basic LCM in > Fuel. We > > >>> conducted a thorough redesign of this feature and splitted it into > several > > >>> granular changes [3]-[6] to allow users to change settings on > deployed, > > >>> partially deployed, stopped or erred clusters and further run > redeployment > > >>> using a particular graph (custom or calculated based on expected > changes > > >>> stored in DB) and with new parameters. > > >>> > > >>> We need 3 weeks after FF to finish this feature. > > >>> Risk of not delivering it after 3 weeks is low. > > >>> > > >>> Patches on review or in progress: > > >>> <https://review.openstack.org/#/c/284139/> > > >>> https://review.openstack.org/#/c/284139/ > > >>> https://review.openstack.org/#/c/279714/ > > >>> https://review.openstack.org/#/c/286754/ > > >>> https://review.openstack.org/#/c/286783/ > > >>> > > >>> Specs: > > >>> https://review.openstack.org/#/c/286713/ > > >>&
[openstack-dev] [Fuel][FFE] Multi-release packages
Fuelers, I would like to request a feature freeze exception for "Multi-release packages" feature [0][1] This feature extends already existing multi-release support in Fuel Plugins. We would like to allow plugin developers to specify all plugin components per release and distribute different deployment graphs, partitioning schemas, network and node roles for each release in one package. This feature is not blocker for us, but it provides very important improvement of users and plugin developers experience. We need 3 weeks after FF to finish this feature. Risk of not delivering it after 3 weeks is low. [0] https://blueprints.launchpad.net/fuel/+spec/plugins-v5 [1] https://review.openstack.org/#/c/271417 --- WBR, Alexey Shtokolov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] [FFE] Unlock Settings Tab
Fuelers, I would like to request a feature freeze exception for "Unlock settings tab" feature [0] This feature being combined with Task-based deployment [1] and LCM-readiness for Fuel deployment tasks [2] unlocks Basic LCM in Fuel. We conducted a thorough redesign of this feature and splitted it into several granular changes [3]-[6] to allow users to change settings on deployed, partially deployed, stopped or erred clusters and further run redeployment using a particular graph (custom or calculated based on expected changes stored in DB) and with new parameters. We need 3 weeks after FF to finish this feature. Risk of not delivering it after 3 weeks is low. Patches on review or in progress: <https://review.openstack.org/#/c/284139/> https://review.openstack.org/#/c/284139/ https://review.openstack.org/#/c/279714/ https://review.openstack.org/#/c/286754/ https://review.openstack.org/#/c/286783/ Specs: https://review.openstack.org/#/c/286713/ https://review.openstack.org/#/c/284797/ https://review.openstack.org/#/c/282695/ https://review.openstack.org/#/c/284250/ [0] https://blueprints.launchpad.net/fuel/+spec/unlock-settings-tab <https://blueprints.launchpad.net/fuel/+spec/unlock-settings-tab>[1] https://blueprints.launchpad.net/fuel/+spec/enable-task-based-deployment [2] https://blueprints.launchpad.net/fuel/+spec/granular-task-lcm-readiness [3] https://blueprints.launchpad.net/fuel/+spec/computable-task-fields-yaql [4] https://blueprints.launchpad.net/fuel/+spec/store-deployment-tasks-history [5] https://blueprints.launchpad.net/fuel/+spec/custom-graph-execution [6] https://blueprints.launchpad.net/fuel/+spec/save-deployment-info-in-database -- --- WBR, Alexey Shtokolov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Task Based Deployment Is at Least Twice Faster
Fuelers, Task based deployment engine has been enabled in master (Fuel 9.0) by default [0] [0] - https://review.openstack.org/#/c/273693/ WBR, Alexey Shtokolov 2016-02-09 21:57 GMT+03:00 Vladimir Kuklin <vkuk...@mirantis.com>: > Folks > > It seems that docker removal spoilt our celebration a bit. Here is a bug > link https://bugs.launchpad.net/fuel/+bug/1543720 . Fix is trivial, but > will postpone swarm run for another day. Nevertheless, it seems to be the > only issue affecting our ability to use TBD. > > Stay tuned! > > On Tue, Feb 9, 2016 at 2:26 PM, Igor Kalnitsky <ikalnit...@mirantis.com> > wrote: > >> > I've run BVT more than 100 times, it works, >> >> You run it some time ago. There were a lot of opportunities to >> introduce regression in both Nailgun and tasks of Fuel Library. ;) >> >> > We are going to run a swarm test today against the ISO with enabled >> task-based deployment >> >> So there will be a custom ISO, correct? If so, it works for me and >> I'll wait for its result. >> >> On Tue, Feb 9, 2016 at 1:17 PM, Alexey Shtokolov >> <ashtoko...@mirantis.com> wrote: >> > Igor, >> > >> > We are going to run a swarm test today against the ISO with enabled >> > task-based deployment, than check results and merge changes tomorrow. >> > I've run BVT more than 100 times, it works, but I would like to check >> more >> > deployment cases. >> > And I guess it should be easy to troubleshoot if docker-related and >> > task-based related changes will be separated by a few days. >> > >> > 2016-02-09 13:39 GMT+03:00 Igor Kalnitsky <ikalnit...@mirantis.com>: >> >> >> >> Well, I'm going to build a new ISO and run BVT. As soon as they are >> >> green, I'm going to approve the change. >> >> >> >> On Tue, Feb 9, 2016 at 12:32 PM, Bogdan Dobrelya < >> bdobre...@mirantis.com> >> >> wrote: >> >> > On 08.02.2016 17:05, Igor Kalnitsky wrote: >> >> >> Hey Fuelers, >> >> >> >> >> >> When we are going to enable it? I think since HCF is passed for >> >> >> stable/8.0, it's time to enable task-based deployment for master >> >> >> branch. >> >> >> >> >> >> Opinion? >> >> > >> >> > This must be done for the 9.0, IMHO. >> >> > >> >> >> >> >> >> - Igor >> >> >> >> >> >> On Wed, Feb 3, 2016 at 12:31 PM, Bogdan Dobrelya >> >> >> <bdobre...@mirantis.com> wrote: >> >> >>> On 02.02.2016 17:35, Alexey Shtokolov wrote: >> >> >>>> Hi Fuelers! >> >> >>>> >> >> >>>> As you may be aware, since [0] Fuel has implemented a new >> >> >>>> orchestration >> >> >>>> engine [1] >> >> >>>> We switched the deployment paradigm from role-based (aka >> granular) to >> >> >>>> task-based and now Fuel can deploy all nodes simultaneously using >> >> >>>> cross-node dependencies between deployment tasks. >> >> >>> >> >> >>> That is great news! Please do not forget about docs updates as >> well. >> >> >>> Those docs are always forgotten like poor orphans... I submitted a >> >> >>> patch >> >> >>> [0] to MOS docs, please review and add more details, if possible, >> for >> >> >>> plugins impact as well. >> >> >>> >> >> >>> [0] https://review.fuel-infra.org/#/c/16509/ >> >> >>> >> >> >>>> >> >> >>>> This feature is experimental in Fuel 8.0 and will be enabled by >> >> >>>> default >> >> >>>> for Fuel 9.0 >> >> >>>> >> >> >>>> Allow me to show you the results. We made some benchmarks on our >> bare >> >> >>>> metal lab [2] >> >> >>>> >> >> >>>> Case #1. 3 controllers + 7 computes w/ ceph. >> >> >>>> Task-based deployment takes *~38* minutes vs *~1h15m* for granular >> >> >>>> (*~2* >> >> >>>> times faster) >> >> >>>> Here and below the deployment time is average time for 10 runs >> >> >>>> >> >
[openstack-dev] [Fuel][Plugins] Multi release packages
Fuelers, We are discussing the idea to extend the multi release packages for plugins. Fuel plugin builder (FPB) can create one rpm-package for all supported releases (from metadata.yaml) but we can specify only deployment scripts and repositories per release. Current release definition (in metadata.yaml): - os: ubuntu version: liberty-8.0 mode: ['ha'] deployment_scripts_path: deployment_scripts/ repository_path: repositories/ubuntu So the idea [0] is to make releases fully configurable. Suggested changes for release definition (in metadata.yaml): components_path: components_liberty.yaml deployment_tasks_path: deployment_tasks_liberty/ # <- folder environment_config_path: environment_config_liberty.yaml network_roles_path: network_roles_liberty.yaml node_roles_path: node_roles_liberty.yaml volumes_path: volumes_liberty.yaml I see the issue: if we change anything for one release (e.g. deployment_task typo) revalidation is needed for all releases. Your Pros and cons please? [0] https://review.openstack.org/#/c/271417/ --- WBR, Alexey Shtokolov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Task Based Deployment Is at Least Twice Faster
Igor,
We are going to run a swarm test today against the ISO with enabled
task-based deployment, than check results and merge changes tomorrow.
I've run BVT more than 100 times, it works, but I would like to check more
deployment cases.
And I guess it should be easy to troubleshoot if docker-related and
task-based related changes will be separated by a few days.
2016-02-09 13:39 GMT+03:00 Igor Kalnitsky <ikalnit...@mirantis.com>:
> Well, I'm going to build a new ISO and run BVT. As soon as they are
> green, I'm going to approve the change.
>
> On Tue, Feb 9, 2016 at 12:32 PM, Bogdan Dobrelya <bdobre...@mirantis.com>
> wrote:
> > On 08.02.2016 17:05, Igor Kalnitsky wrote:
> >> Hey Fuelers,
> >>
> >> When we are going to enable it? I think since HCF is passed for
> >> stable/8.0, it's time to enable task-based deployment for master
> >> branch.
> >>
> >> Opinion?
> >
> > This must be done for the 9.0, IMHO.
> >
> >>
> >> - Igor
> >>
> >> On Wed, Feb 3, 2016 at 12:31 PM, Bogdan Dobrelya <
> bdobre...@mirantis.com> wrote:
> >>> On 02.02.2016 17:35, Alexey Shtokolov wrote:
> >>>> Hi Fuelers!
> >>>>
> >>>> As you may be aware, since [0] Fuel has implemented a new
> orchestration
> >>>> engine [1]
> >>>> We switched the deployment paradigm from role-based (aka granular) to
> >>>> task-based and now Fuel can deploy all nodes simultaneously using
> >>>> cross-node dependencies between deployment tasks.
> >>>
> >>> That is great news! Please do not forget about docs updates as well.
> >>> Those docs are always forgotten like poor orphans... I submitted a
> patch
> >>> [0] to MOS docs, please review and add more details, if possible, for
> >>> plugins impact as well.
> >>>
> >>> [0] https://review.fuel-infra.org/#/c/16509/
> >>>
> >>>>
> >>>> This feature is experimental in Fuel 8.0 and will be enabled by
> default
> >>>> for Fuel 9.0
> >>>>
> >>>> Allow me to show you the results. We made some benchmarks on our bare
> >>>> metal lab [2]
> >>>>
> >>>> Case #1. 3 controllers + 7 computes w/ ceph.
> >>>> Task-based deployment takes *~38* minutes vs *~1h15m* for granular
> (*~2*
> >>>> times faster)
> >>>> Here and below the deployment time is average time for 10 runs
> >>>>
> >>>> Case #2. 3 controllers + 3 mongodb + 4 computes w/ ceph.
> >>>> Task-based deployment takes *~41* minutes vs *~1h32m* for granular
> >>>> (*~2.24* times faster)
> >>>>
> >>>>
> >>>>
> >>>> Also we took measurements for Fuel CI test cases. Standard BVT (Master
> >>>> node + 3 controllers + 3 computes w/ ceph. All are in qemu VMs on one
> host)
> >>>>
> >>>> Fuel CI slaves with *4 *cores *~1.1* times faster
> >>>> In case of 4 cores for 7 VMs they are fighting for CPU resources and
> it
> >>>> marginalizes the gain of task-based deployment
> >>>>
> >>>> Fuel CI slaves with *6* cores *~1.6* times faster
> >>>>
> >>>> Fuel CI slaves with *12* cores *~1.7* times faster
> >>>
> >>> These are really outstanding results!
> >>> (tl;dr)
> >>> I believe the next step may be to leverage the "external install & svc
> >>> management" feature (example [1]) of the Liberty release (7.0.0) of
> >>> Puppet-Openstack (PO) modules. So we could use separate concurrent
> >>> cross-depends based tasks *within a single node* as well, like:
> >>> - task: install_all_packages - a singleton task for a node,
> >>> - task: [configure_x, for each x] - concurrent for a node,
> >>> - task: [manage_service_x, for each x] - some may be concurrent for a
> >>> node, while another shall be serialized.
> >>>
> >>> So, one might use the "--tags" separator for concurrent puppet runs to
> >>> make things go even faster, for example:
> >>>
> >>> # cat test.pp
> >>> notify
> >>> {"A": tag => "a" }
> >>> notify
> >>> {"B": tag => "b" }
> >>>
> >>> # puppet apply test.pp
> >>> Notice: A
> >>> Notice: /Stage[main]/Main/Notify[A]/message:
[openstack-dev] [Fuel] Task Based Deployment Is at Least Twice Faster
Hi Fuelers! As you may be aware, since [0] Fuel has implemented a new orchestration engine [1] We switched the deployment paradigm from role-based (aka granular) to task-based and now Fuel can deploy all nodes simultaneously using cross-node dependencies between deployment tasks. This feature is experimental in Fuel 8.0 and will be enabled by default for Fuel 9.0 Allow me to show you the results. We made some benchmarks on our bare metal lab [2] Case #1. 3 controllers + 7 computes w/ ceph. Task-based deployment takes *~38* minutes vs *~1h15m* for granular (*~2* times faster) Here and below the deployment time is average time for 10 runs Case #2. 3 controllers + 3 mongodb + 4 computes w/ ceph. Task-based deployment takes *~41* minutes vs *~1h32m* for granular (*~2.24* times faster) Also we took measurements for Fuel CI test cases. Standard BVT (Master node + 3 controllers + 3 computes w/ ceph. All are in qemu VMs on one host) Fuel CI slaves with *4 *cores *~1.1* times faster In case of 4 cores for 7 VMs they are fighting for CPU resources and it marginalizes the gain of task-based deployment Fuel CI slaves with *6* cores *~1.6* times faster Fuel CI slaves with *12* cores *~1.7* times faster You can see additional information and charts in the presentation [3]. [0] - http://lists.openstack.org/pipermail/openstack-dev/2015-December/082093.html [1] - https://specs.openstack.org/openstack/fuel-specs/specs/8.0/task-based-deployment-mvp.html [2] - 3 x HP ProLiant DL360p Gen8 (XeonE5 6 cores/64GB/SSD) + 7 x HP ProLiant DL320p Gen8 (XeonE3 4 cores/8-16GB/HDD) [3] - https://docs.google.com/presentation/d/1jZCFZlXHs_VhjtVYS2VuWgdxge5Q6sOMLz4bRLuw7YE --- WBR, Alexey Shtokolov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] PostgreSQL 9.3 and JSON operations
Dmitry, Thank you for this document! Please move it on https://etherpad.openstack.org to make it accessible Best regards, Alexey Shtokolov 2015-12-16 1:38 GMT+03:00 Dmitry Teselkin <dtesel...@mirantis.com>: > Hello, > > I made an attempt to gather all valuable points 'for' and 'against' > 9.2.x in one document [1]. Please take a look on it, I also put some > comments there to keep everything in one place. I believe this can help > us to make deliberated decision. > > Please add more pros / cons there as I don't pretend to make a > full picture at the first attempt. > > Just in case, I'd prefer to 'downgrade' to 9.2 :) > > [1] https://etherpad.mirantis.net/p/7ZUruwlwJM > > On Tue, 15 Dec 2015 20:47:41 +0200 > Igor Kalnitsky <ikalnit...@mirantis.com> wrote: > > > FYI: so far (according to poll [1]) we have > > > > * 11 votes for keeping 9.2 > > * 4 votes for restoring 9.3 > > > > [1] > > > https://docs.google.com/spreadsheets/d/1RNcEVFsg7GdHIXlJl-6LCELhlwQ_zmTbd40Bk_jH1m4/edit?usp=sharing > > > > On Tue, Dec 15, 2015 at 8:34 PM, Vladimir Kuklin > > <vkuk...@mirantis.com> wrote: > > > Folks > > > > > > Let me add my 2c here. > > > > > > I am for using Postgres 9.3. Here is an additional argument to the > > > ones provided by Artem, Aleksandra and others. > > > > > > Fuel is being sometimes highly customized by our users for their > > > specific needs. It has been Postgres 9.3 for a while and they might > > > have as well gotten used to it and assumed by default that this > > > would not change. So some of their respective features they are > > > developing for their own sake may depend on Postgres 9.3 and we > > > will never be able to tell the fraction of such use cases. > > > Moreover, downgrading DBMS version of Fuel should be inevitably > > > considered as a 'deprecation' of some features our software suite > > > is providing to our users. This actually means that we MUST provide > > > our users with a warning and deprecation period to allow them to > > > adjust to these changes. Obviously, accidental change of Postgres > > > version does not follow such a policy in any way. So I see no other > > > ways except for getting back to Postgres 9.3. > > > > > > > > > On Tue, Dec 15, 2015 at 7:39 PM, Igor Kalnitsky > > > <ikalnit...@mirantis.com> wrote: > > >> > > >> Hey Mike, > > >> > > >> Thanks for your input. > > >> > > >> > actually not. if you replace your ARRAY columns with JSON > > >> > entirely, > > >> > > >> It still needs to fix the code, i.e. change ARRAY-specific queries > > >> with JSON ones around the code. ;) > > >> > > >> > there's already a mostly finished PR for SQLAlchemy support in > > >> > the queue. > > >> > > >> Does it mean SQLAlchemy will have one unified interface to make > > >> JSON queries? So we can use different backends if necessary? > > >> > > >> Thanks, > > >> - Igor > > >> > > >> On Tue, Dec 15, 2015 at 5:06 PM, Mike Bayer <mba...@redhat.com> > > >> wrote: > > >> > > > >> > > > >> > On 12/15/2015 07:20 AM, Igor Kalnitsky wrote: > > >> >> Hey Julien, > > >> >> > > >> >>> > > >> >>> > https://blueprints.launchpad.net/fuel/+spec/openstack-ha-fuel-postgresql > > >> >> > > >> >> I believe this blueprint is about DB for OpenStack cloud (we use > > >> >> Galera now), while here we're talking about DB backend for Fuel > > >> >> itself. Fuel has a separate node (so called Fuel Master) and we > > >> >> use PostgreSQL now. > > >> >> > > >> >>> does that mean Fuel is only going to be able to run with > > >> >>> PostgreSQL? > > >> >> > > >> >> Unfortunately we already tied up to PostgreSQL. For instance, > > >> >> we use PostgreSQL's ARRAY column type. Introducing JSON column > > >> >> is one more way to tighten knots harder. > > >> > > > >> > actually not. if you replace your ARRAY columns with JSON > > >> > entirely, MySQL has JSON as well now: > > >> > https://dev.mysql.com/doc/refman/5.7/en/json.ht
Re: [openstack-dev] [Fuel] PostgreSQL 9.3 and JSON operations
On Tue, Dec 15, 2015 at 9:47 PM, Igor Kalnitsky <ikalnit...@mirantis.com> wrote: > * 11 votes for keeping 9.2 > * 4 votes for restoring 9.3 Igor, please remove my vote from "9.2", I voted for "I'm too conservative, I want to see classic RDBMS approach" , but not to keep accidentally downgraded PostgreSQL If you're asking about using JSON in our PostgreSQL - no, it's not obligatory, but we can discuss it for specific cases (IMO it's like "syntactic sugar" for RDB). If you're asking: should we unexpectedly downgrade DB version after FF and make upgrade procedure more complicated - strongly disagree. The reasons are well described above (in Alexandra's mail): > * It is clear that PostgreSQL downgrade wasn't planned and discussed > before Feature Freeze, so this change is accidental. We didn't > investigate all possible consequences and changes required for the > switch. > * In Infra we have all our unit tests run on PostgreSQL 9.3. > * For Maintenance team it adds the burden of supporting yet another > version while they have PostgreSQL 9.3 anyway. So this change doesn't > reduce number of supported versions, it rather adds one to the list. So I think we should keep 9.3 and continue this discussion in the beginning of 9.0 release. Best regards, Alexey Shtokolov 2015-12-15 22:58 GMT+03:00 Vitaly Kramskikh <vkramsk...@mirantis.com>: > +1 to Vova and Sasha, > > I voted for 9.2 at the beginning of the thread due to potential packaging > and infrastructure issues, but since Artem and Sasha insist on 9.3, I see > no reasons to keep 9.2. > > 2015-12-15 22:19 GMT+03:00 Aleksandra Fedorova <afedor...@mirantis.com>: > >> Igor, >> >> that's an anonymous vote for question stated in a wrong way. Sorry, >> but it doesn't really look like a valuable input for the discussion. >> >> On Tue, Dec 15, 2015 at 9:47 PM, Igor Kalnitsky <ikalnit...@mirantis.com> >> wrote: >> > FYI: so far (according to poll [1]) we have >> > >> > * 11 votes for keeping 9.2 >> > * 4 votes for restoring 9.3 >> > >> > [1] >> https://docs.google.com/spreadsheets/d/1RNcEVFsg7GdHIXlJl-6LCELhlwQ_zmTbd40Bk_jH1m4/edit?usp=sharing >> > >> > On Tue, Dec 15, 2015 at 8:34 PM, Vladimir Kuklin <vkuk...@mirantis.com> >> wrote: >> >> Folks >> >> >> >> Let me add my 2c here. >> >> >> >> I am for using Postgres 9.3. Here is an additional argument to the ones >> >> provided by Artem, Aleksandra and others. >> >> >> >> Fuel is being sometimes highly customized by our users for their >> specific >> >> needs. It has been Postgres 9.3 for a while and they might have as well >> >> gotten used to it and assumed by default that this would not change. >> So some >> >> of their respective features they are developing for their own sake may >> >> depend on Postgres 9.3 and we will never be able to tell the fraction >> of >> >> such use cases. Moreover, downgrading DBMS version of Fuel should be >> >> inevitably considered as a 'deprecation' of some features our software >> suite >> >> is providing to our users. This actually means that we MUST provide our >> >> users with a warning and deprecation period to allow them to adjust to >> these >> >> changes. Obviously, accidental change of Postgres version does not >> follow >> >> such a policy in any way. So I see no other ways except for getting >> back to >> >> Postgres 9.3. >> >> >> >> >> >> On Tue, Dec 15, 2015 at 7:39 PM, Igor Kalnitsky < >> ikalnit...@mirantis.com> >> >> wrote: >> >>> >> >>> Hey Mike, >> >>> >> >>> Thanks for your input. >> >>> >> >>> > actually not. if you replace your ARRAY columns with JSON entirely, >> >>> >> >>> It still needs to fix the code, i.e. change ARRAY-specific queries >> >>> with JSON ones around the code. ;) >> >>> >> >>> > there's already a mostly finished PR for SQLAlchemy support in the >> >>> > queue. >> >>> >> >>> Does it mean SQLAlchemy will have one unified interface to make JSON >> >>> queries? So we can use different backends if necessary? >> >>> >> >>> Thanks, >> >>> - Igor >> >>> >> >>> On Tue, Dec 15, 2015 at 5:06 PM, Mike Bayer <mba...@redhat.com> >> wrote: >> >>> > >>
Re: [openstack-dev] [Fuel][CI] recheck/reverify support for Fuel CI jobs
Igor, Thank you for this feature. Afaiu recheck/reverify is mostly useful for internal CI-related fails. And Fuel CI and Openstack CI are two different infrastructures. So if smth is broken on Fuel CI, "recheck" will restart all jobs on Openstack CI too. And opposite case works the same way. Probably we should use another keyword for Fuel CI to prevent an extra load on the infrastructure? For example "refuel" or smth like this? Best regards, Alexey Shtokolov 2015-11-20 14:24 GMT+03:00 Stanislaw Bogatkin <sbogat...@mirantis.com>: > Igor, > > it is much more clear for me now. Thank you :) > > On Fri, Nov 20, 2015 at 2:09 PM, Igor Belikov <ibeli...@mirantis.com> > wrote: > >> Hi Stanislaw, >> >> The reason behind this is simple - deployment tests are heavy. Each >> deployment test occupies whole server for ~2 hours, for each commit we have >> 2 deployment tests (for current fuel-library master) and that’s just >> because we don’t test CentOS deployment for now. >> If we assume that developers will rertrigger deployment tests only when >> retrigger would actually solve the failure - it’s still not smart in terms >> of HW usage to retrigger both tests when only one has failed, for example. >> And there are cases when retrigger just won’t do it and CI Engineer must >> manually erase the existing environment on slave or fix it by other means, >> so it’s better when CI Engineer looks through logs before each retrigger of >> deployment test. >> >> Hope this answers your question. >> >> -- >> Igor Belikov >> Fuel CI Engineer >> ibeli...@mirantis.com >> >> On 20 Nov 2015, at 13:57, Stanislaw Bogatkin <sbogat...@mirantis.com> >> wrote: >> >> Hi Igor, >> >> would you be so kind tell, why fuel-library deployment tests doesn't >> support this? Maybe there is a link with previous talks about it? >> >> On Fri, Nov 20, 2015 at 1:34 PM, Igor Belikov <ibeli...@mirantis.com> >> wrote: >> >>> Hi, >>> >>> I’d like to inform you that all jobs running on Fuel CI (with the >>> exception of fuel-library deployment tests) now support retriggering via >>> “recheck” or “reverify” comments in Gerrit. >>> Exact regex is the same one used in Openstack-Infra’s zuul and can be >>> found here >>> https://github.com/fuel-infra/jenkins-jobs/blob/master/servers/fuel-ci/global.yaml#L3 >>> >>> CI-Team kindly asks you to not abuse this option, unfortunately not >>> every failure could be solved by retriggering. >>> And, to stress this out once again: fuel-library deployment tests don’t >>> support this, so you still have to ask for a retrigger in #fuel-infra irc >>> channel. >>> >>> Thanks for attention. >>> -- >>> Igor Belikov >>> Fuel CI Engineer >>> ibeli...@mirantis.com >>> >>> >>> >>> >>> >>> >>> >>> >>> __ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> <http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org >> ?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- --- WBR, Alexey Shtokolov __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
