Re: [openstack-dev] [fuel][tripleo] Fuel and TripleO Cross-Project Integration

[Aleksandr Didenko]

Hi,

> 3. Consider integration of Fuel provisioning, network configuration and
disk partitioning with Ironic and TripleO

As a part of this, I'd like to propose "l23network" puppet module [0]
separation from fuel-library. This would allow anyone to reuse it for
software-defined network configuration of OpenStack nodes. This module
already supports pretty complex configurations, such as bonds (including
LACP and other bond properties), OVS, SR-IOV, OVS-DPDK, etc.

Regards,
Alex

[0]
https://github.com/openstack/fuel-library/tree/master/deployment/puppet/l23network

On Wed, Nov 9, 2016 at 3:37 PM, Vladimir Kuklin 
wrote:

> Stackers
>
> During OpenStack summit in Barcelona we had a rather productive
> cross-project integration session between Fuel and TripleO projects [0].
> More precisely we discussed the following:
>
> 1. Services composition and deployment workflow control
> 2. Complex networking configuration (bonds, offloading, etc.)
> 3. Post-deployment cluster management, i.e. day-2 operations
> 4. Approaches to reference architecture recomposition, e.g. in-flight
> database vertical scaling
> 5. Upgrades
> 6. Provisioning and disks partitioning
>
> The most interesting part of integration would be to consider integration
> of deployment and post-deployment orchestration approaches and services
> composition. The further steps are to start working on:
>
> 1. Cross-project specification on workflow management, e.g. for upgrades
> and day-2 operations
> 2. Look into possibility of Mistral integration as a workflow execution
> tool within the designed model
> 3. Consider integration of Fuel provisioning, network configuration and
> disk partitioning with Ironic and TripleO
>
> [0] https://etherpad.openstack.org/p/fuel-ocata-fuel-tripleo-integration
>
> I would suggest that we add discussing this into TripleO or Fuel IRC
> meeting agenda next week.
>
> Please let me know what are you thoughts on this.
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com 
> www.mirantis.ru
> vkuk...@mirantis.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel-library] Nominate Stanislaw Bogatkin to fuel-library core

[Aleksandr Didenko]

+1

On Thu, Sep 8, 2016 at 11:06 AM, Sergey Vasilenko 
wrote:

> +1
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Common fuel-core group for all Fuel projects

[Aleksandr Didenko]

Hi,

-1 for the proposal

Regards,
Alex

On Tue, Sep 6, 2016 at 10:42 AM, Alexey Stepanov 
wrote:

> Guys, I have one serious question: WHO will be global core?
> Example:
>   I'm core reviewer in 2 repos, but I'm absolutely could not be core
> reviewer in puppet!
>
> On Tue, Sep 6, 2016 at 11:31 AM, Igor Kalnitsky 
> wrote:
>
>> -1 for the proposal. I see no problems to add guys who're familiar
>> with various sub-projects to multiple core groups.
>>
>> On Tue, Sep 6, 2016 at 10:38 AM, Evgeniy L  wrote:
>> > +1 to Lukasz.
>> > -1 to the proposal, we had it this way for a quite some time, and it
>> was not
>> > good for the project (as Lukasz pointed out), why should a person who
>> merges
>> > the code to the library have an access to merge the code to
>> Nailgun/Astute
>> > without proper expertise. Those are different areas which require
>> different
>> > skills.
>> >
>> > Thanks,
>> >
>> > On Tue, Sep 6, 2016 at 9:40 AM, Lukasz Oles  wrote:
>> >>
>> >> On Mon, Sep 5, 2016 at 5:39 PM, Andrew Maksimov <
>> amaksi...@mirantis.com>
>> >> wrote:
>> >> > +1
>> >> > This is a good proposal, I also think we should have single fuel-core
>> >> > group
>> >> > for all repos. In real life core reviewers won't set +2 or merge to
>> >> > repos
>> >> > with which they are not familiar with.
>> >> Actually one of the reasons why core groups were split was that it
>> >> happened a few times :)
>> >>
>> >> >
>> >> > Regards,
>> >> > Andrey Maximov
>> >> >
>> >> > On Mon, Sep 5, 2016 at 2:11 PM, Vladimir Kozhukalov
>> >> >  wrote:
>> >> >>
>> >> >> Dear colleagues,
>> >> >>
>> >> >> I'd like to suggest to use common fuel-core group for all Fuel
>> projects
>> >> >> instead of having separate independent 'by-project' core groups like
>> >> >> 'fuel-astute-core' or 'fuel-agent-core'.
>> >> >>
>> >> >> Pros:
>> >> >> 1) It will be easier to access core members (timezone and holiday
>> >> >> tolerance)
>> >> >> 2) It will be easier to manage single core group (promote new
>> members,
>> >> >> remove not active members)
>> >> >>
>> >> >> Cons:
>> >> >> 1) Less of flexibility. Permissions will be the same for all core
>> >> >> reviewers in all Fuel projects.
>> >> >>
>> >> >> What do you think?
>> >> >>
>> >> >> Vladimir Kozhukalov
>> >> >>
>> >> >>
>> >> >> 
>> __
>> >> >> OpenStack Development Mailing List (not for usage questions)
>> >> >> Unsubscribe:
>> >> >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > 
>> __
>> >> > OpenStack Development Mailing List (not for usage questions)
>> >> > Unsubscribe:
>> >> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Łukasz Oleś
>> >>
>> >> 
>> __
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe: openstack-dev-requ...@lists.op
>> enstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >
>> > 
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: openstack-dev-requ...@lists.op
>> enstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Alexey Stepanov
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] Propose Denis Egorenko for fuel-library core

[Aleksandr Didenko]

+1

On Thu, Aug 25, 2016 at 9:35 AM, Sergey Vasilenko 
wrote:

> +1
>
>
> /sv
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] It is impossible to queue UpdateDnsmasqTask

[Aleksandr Didenko]

Hi,

well, we have only one DHCP server that serves multiple clusters. Actions
with those multiple clusters may affect DHCP server configuration. So
queueing tasks that change DHCP server configuration seems like a
reasonable way to fix the problem. So options 2 and 3 are much better than
1 or 4.

Regards,
Alex

On Thu, Jul 7, 2016 at 10:59 AM, Georgy Kibardin 
wrote:

> Continuing speaking to myself :)
>
> Option 4 implies that we generate puppet manifest with a set of admin
> networks instead of writing it into hiera. So, we get a two step task
> which, first, generates manifest with unique name and, second, calls puppet
> to apply it.
> However, theres still a problem with this approach. When we almost
> simultaneously delete environments A and B (and A goes a bit earlier)
> astute acknowledges two UpdateDnsmasqTask tasks for execution, however, it
> cannot guarantee that puppet for UpdateDnsmasqTask for env A will be
> executed earlier than for env B. This would lead to incorrect list of admin
> networks by the end of environment deletion.
>
> So the option 4 just doesn't work.
>
> On Wed, Jul 6, 2016 at 11:41 AM, Georgy Kibardin 
> wrote:
>
>> Bulat is suggesting to move with 4. He suggest to merge all actions of
>> UpdateDnsmasqTask into one puppet task. There are three actions: syncing
>> admin network list to heira, dhcp ranges update and cobbler sync. The
>> problem I see with this approach is that current implementation does not
>> suppose passing any additional data to "puppet apply". Cobbler sync seems
>> to be a reasonable part of updating dhcp ranges config.
>>
>> Best,
>> Georgy
>>
>> On Thu, Jun 16, 2016 at 7:25 PM, Georgy Kibardin 
>> wrote:
>>
>>> Hi All,
>>>
>>> Currently we can only run one instance of subj. at time. An attempt to
>>> run second one causes an exception. This behaviour at least may cause a
>>> cluster to stuck forever in "removing" state (reproduces here
>>> https://bugs.launchpad.net/fuel/+bug/1544493) or just produce
>>> incomprehensible "task already running" message. So we need to address the
>>> problem somehow. I see the following ways to fix it:
>>>
>>> 1. Just put the cluster into "error" state which would allow user to
>>> remove it later.
>>>   pros: simple and fixes the problem at hand (#1544493)
>>>   cons: it would be hard to detect "come againg later" situation; quite
>>> a lame behavior: why don't you "come again later" yourself.
>>>
>>> 2. Implement generic queueing in nailgun.
>>> pros: quite simple
>>> cons: it doesn't look like nailgun responsibility
>>>
>>> 3. Implement generic queueing in astute.
>>>pros: this behaviour makes sense for astute.
>>>cons: the implementation would be quite complex, we need to
>>> synchronize execution between separate worker processes.
>>>
>>> 4. Split the task so that each part would work with particular cluster.
>>>pros: we don't extend our execution model
>>>cons: untrivial implementation; no guarantee that we are always able
>>> to split master node tasks on per cluster basis.
>>>
>>> Best,
>>> Georgy
>>>
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] - Nominate Maksim Malchuk to Fuel Library Core

[Aleksandr Didenko]

Hi,

+1 from me.

Regards,
Alex

On Mon, Jun 27, 2016 at 4:54 PM, Sergii Golovatiuk  wrote:

> I am very sorry for sending without subject. I am adding subject to voting
> and my +1
>
> --
> Best regards,
> Sergii Golovatiuk,
> Skype #golserge
> IRC #holser
>
> On Mon, Jun 27, 2016 at 4:42 PM, Sergii Golovatiuk <
> sgolovat...@mirantis.com> wrote:
>
>> Hi,
>>
>> I would like to nominate Maksim Malchuk to Fuel-Library Core team. He’s
>> been doing a great job so far [0]. He’s #2 reviewer and #2 contributor with
>> 28 commits for last 90 days [1][2].
>>
>> Fuelers, please vote with +1/-1 for approval/objection. Voting will be
>> open until July of 4th. This will go forward after voting is closed if
>> there are no objections.
>>
>> Overall contribution:
>> [0] http://stackalytics.com/?user_id=mmalchuk
>> Fuel library contribution for last 90 days:
>> [1]  
>> http://stackalytics.com/report/contribution/fuel-library/90
>> http://stackalytics.com/report/users/mmalchuk
>> List of reviews:
>> [2]
>> https://review.openstack.org/#/q/reviewer:%22Maksim+Malchuk%22+status:merged,n,z
>> --
>> Best regards,
>> Sergii Golovatiuk,
>> Skype #golserge
>> IRC #holser
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] [Plugins] Netconfig tasks changes

[Aleksandr Didenko]

Hi folks,

it turned out that one change has to be made in your plugins though. If
you're reusing 'netconfig' task for your custom roles, you need to remove
'configure_default_route' task from your roles (if you have it) and add
'hiera_default_route'. This is needed for proper default gateway
configuration for cases when public network is assigned on controllers
only. Here's an example of patch for plugin [0] and bug about it [1]

Regards,
Alex

[0] https://review.openstack.org/#/c/328225/2/deployment_tasks.yaml
[1] https://bugs.launchpad.net/fuel-plugins/+bug/1591117

On Tue, Jun 7, 2016 at 11:46 AM, Aleksandr Didenko <adide...@mirantis.com>
wrote:

> Hi,
>
> you don't need to change anything in your plugin, we still have the same
> netconfig.pp task on all nodes even after bugfix.
>
> Regards,
> Alex
>
> On Tue, Jun 7, 2016 at 8:21 AM, Igor Zinovik <izino...@mirantis.com>
> wrote:
>
>>   Hello,
>>
>> Aleksandr, one simple question: do I as a plugin developer for upcoming
>> Fuel 9.0 have
>> to worry about these network-related changes or not? HCF is approaching,
>> but patch
>> that you mentioned (342307 <https://review.openstack.org/#/c/324307/>)
>> is still not merged. Do I need to spend time on understanding
>> it and change plugins deployment tasks
>> <https://github.com/openstack/fuel-plugin-nsxv/blob/bab9541d9f13cf80a4796117483e56e94f3a4c09/deployment_tasks.yaml#L1-L10>
>> according to the netconfig.pp refactoring?
>>
>>
>> On 6 June 2016 at 11:12, Aleksandr Didenko <adide...@mirantis.com> wrote:
>>
>>> Hi,
>>>
>>> a bit different patch is on review now [0]. Instead of silently
>>> replacing default gateway on the fly in netconfig.pp task it's putting new
>>> default gateway into Hiera. Thus we'll have idempotency for subsequent
>>> netconfig.pp runs even on Mongo roles. Also we'll have consistent network
>>> configuration data in Hiera which any plugin can rely on.
>>>
>>> I've built a custom ISO with this patch and run a set of custom tests on
>>> it to cover multi-role and multi-rack cases [1] plus BVT - everything
>>> worked fine.
>>>
>>> Please feel free to review and comment the patch [0].
>>>
>>> Regards,
>>> Alex
>>>
>>> [0] https://review.openstack.org/324307
>>> [1] http://paste.openstack.org/show/508319/
>>>
>>> On Wed, Jun 1, 2016 at 4:47 PM, Aleksandr Didenko <adide...@mirantis.com
>>> > wrote:
>>>
>>>> Hi,
>>>>
>>>> YAQL expressions support for task dependencies has been added to
>>>> Nailgun [0]. So now it's possible to fix network configuration idempotency
>>>> issue without introducing new 'netconfig' task [1]. There will be no
>>>> problems with loops in task graph in such case (tested on multiroles,
>>>> worked fine). When we deprecate role-based deployment (even emulated), then
>>>> we'll be able to remove all those additional conditions from manifests and
>>>> remove 'configure_default_route' task completely. Please feel free to
>>>> review and comment the patch [1].
>>>>
>>>> Regards,
>>>> Alex
>>>>
>>>> [0] https://review.openstack.org/#/c/320861/
>>>> [1] https://review.openstack.org/#/c/322872/
>>>>
>>>> On Wed, May 25, 2016 at 10:39 AM, Simon Pasquier <
>>>> spasqu...@mirantis.com> wrote:
>>>>
>>>>> Hi Adam,
>>>>> Maybe you want to look into network templates [1]? Although the
>>>>> documentation is a bit sparse, it allows you to define flexible network
>>>>> mappings.
>>>>> BR,
>>>>> Simon
>>>>> [1]
>>>>> https://docs.mirantis.com/openstack/fuel/fuel-8.0/operations.html#using-networking-templates
>>>>>
>>>>> On Wed, May 25, 2016 at 10:26 AM, Adam Heczko <ahec...@mirantis.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks Alex, will experiment with it once again although AFAIR it
>>>>>> doesn't solve thing I'd like to do.
>>>>>> I'll come later to you in case of any questions.
>>>>>>
>>>>>>
>>>>>> On Wed, May 25, 2016 at 10:00 AM, Aleksandr Didenko <
>>>>>> adide...@mirantis.com> wrote:
>>>>>>
>>>>>>> Hey Adam,
>>>>>>>
>>>>>>> in Fuel we have the following option (checkbox) on Network Se

Re: [openstack-dev] [Fuel] [Plugins] Netconfig tasks changes

[Aleksandr Didenko]

Hi,

you don't need to change anything in your plugin, we still have the same
netconfig.pp task on all nodes even after bugfix.

Regards,
Alex

On Tue, Jun 7, 2016 at 8:21 AM, Igor Zinovik <izino...@mirantis.com> wrote:

>   Hello,
>
> Aleksandr, one simple question: do I as a plugin developer for upcoming
> Fuel 9.0 have
> to worry about these network-related changes or not? HCF is approaching,
> but patch
> that you mentioned (342307 <https://review.openstack.org/#/c/324307/>) is
> still not merged. Do I need to spend time on understanding
> it and change plugins deployment tasks
> <https://github.com/openstack/fuel-plugin-nsxv/blob/bab9541d9f13cf80a4796117483e56e94f3a4c09/deployment_tasks.yaml#L1-L10>
> according to the netconfig.pp refactoring?
>
>
> On 6 June 2016 at 11:12, Aleksandr Didenko <adide...@mirantis.com> wrote:
>
>> Hi,
>>
>> a bit different patch is on review now [0]. Instead of silently replacing
>> default gateway on the fly in netconfig.pp task it's putting new default
>> gateway into Hiera. Thus we'll have idempotency for subsequent netconfig.pp
>> runs even on Mongo roles. Also we'll have consistent network configuration
>> data in Hiera which any plugin can rely on.
>>
>> I've built a custom ISO with this patch and run a set of custom tests on
>> it to cover multi-role and multi-rack cases [1] plus BVT - everything
>> worked fine.
>>
>> Please feel free to review and comment the patch [0].
>>
>> Regards,
>> Alex
>>
>> [0] https://review.openstack.org/324307
>> [1] http://paste.openstack.org/show/508319/
>>
>> On Wed, Jun 1, 2016 at 4:47 PM, Aleksandr Didenko <adide...@mirantis.com>
>> wrote:
>>
>>> Hi,
>>>
>>> YAQL expressions support for task dependencies has been added to Nailgun
>>> [0]. So now it's possible to fix network configuration idempotency issue
>>> without introducing new 'netconfig' task [1]. There will be no problems
>>> with loops in task graph in such case (tested on multiroles, worked fine).
>>> When we deprecate role-based deployment (even emulated), then we'll be able
>>> to remove all those additional conditions from manifests and remove
>>> 'configure_default_route' task completely. Please feel free to review and
>>> comment the patch [1].
>>>
>>> Regards,
>>> Alex
>>>
>>> [0] https://review.openstack.org/#/c/320861/
>>> [1] https://review.openstack.org/#/c/322872/
>>>
>>> On Wed, May 25, 2016 at 10:39 AM, Simon Pasquier <spasqu...@mirantis.com
>>> > wrote:
>>>
>>>> Hi Adam,
>>>> Maybe you want to look into network templates [1]? Although the
>>>> documentation is a bit sparse, it allows you to define flexible network
>>>> mappings.
>>>> BR,
>>>> Simon
>>>> [1]
>>>> https://docs.mirantis.com/openstack/fuel/fuel-8.0/operations.html#using-networking-templates
>>>>
>>>> On Wed, May 25, 2016 at 10:26 AM, Adam Heczko <ahec...@mirantis.com>
>>>> wrote:
>>>>
>>>>> Thanks Alex, will experiment with it once again although AFAIR it
>>>>> doesn't solve thing I'd like to do.
>>>>> I'll come later to you in case of any questions.
>>>>>
>>>>>
>>>>> On Wed, May 25, 2016 at 10:00 AM, Aleksandr Didenko <
>>>>> adide...@mirantis.com> wrote:
>>>>>
>>>>>> Hey Adam,
>>>>>>
>>>>>> in Fuel we have the following option (checkbox) on Network Setting
>>>>>> tab:
>>>>>>
>>>>>> Assign public network to all nodes
>>>>>> When disabled, public network will be assigned to controllers only
>>>>>>
>>>>>> So if you uncheck it (by default it's unchecked) then public network
>>>>>> and 'br-ex' will exist on controllers only. Other nodes won't even have
>>>>>> "Public" network on node interface configuration UI.
>>>>>>
>>>>>> Regards,
>>>>>> Alex
>>>>>>
>>>>>> On Wed, May 25, 2016 at 9:43 AM, Adam Heczko <ahec...@mirantis.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello Alex,
>>>>>>> I have a question about the proposed changes.
>>>>>>> Is it possible to introduce new vlan and associated bridge only for
>>>>>>> contr

Re: [openstack-dev] [Fuel] [Plugins] Netconfig tasks changes

[Aleksandr Didenko]

Hi,

a bit different patch is on review now [0]. Instead of silently replacing
default gateway on the fly in netconfig.pp task it's putting new default
gateway into Hiera. Thus we'll have idempotency for subsequent netconfig.pp
runs even on Mongo roles. Also we'll have consistent network configuration
data in Hiera which any plugin can rely on.

I've built a custom ISO with this patch and run a set of custom tests on it
to cover multi-role and multi-rack cases [1] plus BVT - everything worked
fine.

Please feel free to review and comment the patch [0].

Regards,
Alex

[0] https://review.openstack.org/324307
[1] http://paste.openstack.org/show/508319/

On Wed, Jun 1, 2016 at 4:47 PM, Aleksandr Didenko <adide...@mirantis.com>
wrote:

> Hi,
>
> YAQL expressions support for task dependencies has been added to Nailgun
> [0]. So now it's possible to fix network configuration idempotency issue
> without introducing new 'netconfig' task [1]. There will be no problems
> with loops in task graph in such case (tested on multiroles, worked fine).
> When we deprecate role-based deployment (even emulated), then we'll be able
> to remove all those additional conditions from manifests and remove
> 'configure_default_route' task completely. Please feel free to review and
> comment the patch [1].
>
> Regards,
> Alex
>
> [0] https://review.openstack.org/#/c/320861/
> [1] https://review.openstack.org/#/c/322872/
>
> On Wed, May 25, 2016 at 10:39 AM, Simon Pasquier <spasqu...@mirantis.com>
> wrote:
>
>> Hi Adam,
>> Maybe you want to look into network templates [1]? Although the
>> documentation is a bit sparse, it allows you to define flexible network
>> mappings.
>> BR,
>> Simon
>> [1]
>> https://docs.mirantis.com/openstack/fuel/fuel-8.0/operations.html#using-networking-templates
>>
>> On Wed, May 25, 2016 at 10:26 AM, Adam Heczko <ahec...@mirantis.com>
>> wrote:
>>
>>> Thanks Alex, will experiment with it once again although AFAIR it
>>> doesn't solve thing I'd like to do.
>>> I'll come later to you in case of any questions.
>>>
>>>
>>> On Wed, May 25, 2016 at 10:00 AM, Aleksandr Didenko <
>>> adide...@mirantis.com> wrote:
>>>
>>>> Hey Adam,
>>>>
>>>> in Fuel we have the following option (checkbox) on Network Setting tab:
>>>>
>>>> Assign public network to all nodes
>>>> When disabled, public network will be assigned to controllers only
>>>>
>>>> So if you uncheck it (by default it's unchecked) then public network
>>>> and 'br-ex' will exist on controllers only. Other nodes won't even have
>>>> "Public" network on node interface configuration UI.
>>>>
>>>> Regards,
>>>> Alex
>>>>
>>>> On Wed, May 25, 2016 at 9:43 AM, Adam Heczko <ahec...@mirantis.com>
>>>> wrote:
>>>>
>>>>> Hello Alex,
>>>>> I have a question about the proposed changes.
>>>>> Is it possible to introduce new vlan and associated bridge only for
>>>>> controllers?
>>>>> I think about DMZ use case and possibility to expose public IPs/VIP
>>>>> and API endpoints on controllers on a completely separate L2 network
>>>>> (segment vlan/bridge) not present on any other nodes than controllers.
>>>>> Thanks.
>>>>>
>>>>> On Wed, May 25, 2016 at 9:28 AM, Aleksandr Didenko <
>>>>> adide...@mirantis.com> wrote:
>>>>>
>>>>>> Hi folks,
>>>>>>
>>>>>> we had to revert those changes [0] since it's impossible to propery
>>>>>> handle two different netconfig tasks for multi-role nodes. So everything
>>>>>> stays as it was before - we have single task 'netconfig' to configure
>>>>>> network for all roles and you don't need to change anything in your
>>>>>> plugins. Sorry for inconvenience.
>>>>>>
>>>>>> Our current plan for fixing network idempotency is to keep one task
>>>>>> but change 'cross-depends' parameter to yaql_exp. This will allow us to 
>>>>>> use
>>>>>> single 'netconfig' task for all roles but at the same time we'll be able 
>>>>>> to
>>>>>> properly order it: netconfig on non-controllers will be executed only
>>>>>> aftetr 'virtual_ips' task.
>>>>>>
>>>>>> Regards,
>>>&

Re: [openstack-dev] [Fuel] [Plugins] Netconfig tasks changes

[Aleksandr Didenko]

Hi,

YAQL expressions support for task dependencies has been added to Nailgun
[0]. So now it's possible to fix network configuration idempotency issue
without introducing new 'netconfig' task [1]. There will be no problems
with loops in task graph in such case (tested on multiroles, worked fine).
When we deprecate role-based deployment (even emulated), then we'll be able
to remove all those additional conditions from manifests and remove
'configure_default_route' task completely. Please feel free to review and
comment the patch [1].

Regards,
Alex

[0] https://review.openstack.org/#/c/320861/
[1] https://review.openstack.org/#/c/322872/

On Wed, May 25, 2016 at 10:39 AM, Simon Pasquier <spasqu...@mirantis.com>
wrote:

> Hi Adam,
> Maybe you want to look into network templates [1]? Although the
> documentation is a bit sparse, it allows you to define flexible network
> mappings.
> BR,
> Simon
> [1]
> https://docs.mirantis.com/openstack/fuel/fuel-8.0/operations.html#using-networking-templates
>
> On Wed, May 25, 2016 at 10:26 AM, Adam Heczko <ahec...@mirantis.com>
> wrote:
>
>> Thanks Alex, will experiment with it once again although AFAIR it doesn't
>> solve thing I'd like to do.
>> I'll come later to you in case of any questions.
>>
>>
>> On Wed, May 25, 2016 at 10:00 AM, Aleksandr Didenko <
>> adide...@mirantis.com> wrote:
>>
>>> Hey Adam,
>>>
>>> in Fuel we have the following option (checkbox) on Network Setting tab:
>>>
>>> Assign public network to all nodes
>>> When disabled, public network will be assigned to controllers only
>>>
>>> So if you uncheck it (by default it's unchecked) then public network and
>>> 'br-ex' will exist on controllers only. Other nodes won't even have
>>> "Public" network on node interface configuration UI.
>>>
>>> Regards,
>>> Alex
>>>
>>> On Wed, May 25, 2016 at 9:43 AM, Adam Heczko <ahec...@mirantis.com>
>>> wrote:
>>>
>>>> Hello Alex,
>>>> I have a question about the proposed changes.
>>>> Is it possible to introduce new vlan and associated bridge only for
>>>> controllers?
>>>> I think about DMZ use case and possibility to expose public IPs/VIP and
>>>> API endpoints on controllers on a completely separate L2 network (segment
>>>> vlan/bridge) not present on any other nodes than controllers.
>>>> Thanks.
>>>>
>>>> On Wed, May 25, 2016 at 9:28 AM, Aleksandr Didenko <
>>>> adide...@mirantis.com> wrote:
>>>>
>>>>> Hi folks,
>>>>>
>>>>> we had to revert those changes [0] since it's impossible to propery
>>>>> handle two different netconfig tasks for multi-role nodes. So everything
>>>>> stays as it was before - we have single task 'netconfig' to configure
>>>>> network for all roles and you don't need to change anything in your
>>>>> plugins. Sorry for inconvenience.
>>>>>
>>>>> Our current plan for fixing network idempotency is to keep one task
>>>>> but change 'cross-depends' parameter to yaql_exp. This will allow us to 
>>>>> use
>>>>> single 'netconfig' task for all roles but at the same time we'll be able 
>>>>> to
>>>>> properly order it: netconfig on non-controllers will be executed only
>>>>> aftetr 'virtual_ips' task.
>>>>>
>>>>> Regards,
>>>>> Alex
>>>>>
>>>>> [0] https://review.openstack.org/#/c/320530/
>>>>>
>>>>>
>>>>> On Thu, May 19, 2016 at 2:36 PM, Aleksandr Didenko <
>>>>> adide...@mirantis.com> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> please be aware that now we have two netconfig tasks (in Fuel 9.0+):
>>>>>>
>>>>>> - netconfig-controller - executed on controllers only
>>>>>> - netconfig - executed on all other nodes
>>>>>>
>>>>>> puppet manifest is the same, but tasks are different. We had to do
>>>>>> this [0] in order to fix network idempotency issues [1].
>>>>>>
>>>>>> So if you have 'netconfig' requirements in your plugin's tasks,
>>>>>> please make sure to add 'netconfig-controller' as well, to work properly 
>>>>>> on
>>>>>> controllers.
>>>>>>
>>>>>> Regards,
>>>&

Re: [openstack-dev] [fuel] discovery and deploy a compute node automatically

[Aleksandr Didenko]

Hi,

+1 to Igor. It should be easily doable via some sort of "watcher" script
(run it as a daemon or under cron), that script should:

- watch for new nodes in 'discover' state. CLI example:
  fuel nodes
- assign new nodes to env with compute role. CLI example:
  fuel --env $ENV_ID node set --node $NEW_NODE_ID --role compute
- update networks assignment for new node. CLI example:
  fuel node --node $NEW_NODE_ID --network --download
  # edit /root/node_$NEW_NODE_ID/interfaces.yaml
  fuel node --node $NEW_NODE_ID --network --upload
- deploy changes. CLI example:
  fuel deploy-changes --env $ENV_ID

Regards,
Alex

On Wed, May 25, 2016 at 12:03 PM, Igor Kalnitsky 
wrote:

> Hey Jason,
>
> What do you mean by "automatically"?
>
> You need to assign "compute" role on that discovered node, and hit "Deploy
> Changes" button. If you really want to deploy any new discovered node
> automatically, I think you can create some automation script and put it
> under cron.
>
> Hope it helps,
> Igor
>
> > On May 25, 2016, at 12:33, jason  wrote:
> >
> > Hi All,
> >
> > Is there any way for fuel to deploy a newly discovered node as a compute
> node automatically? I followed the openstack doc for fuel but did not get
> any answer.
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] [Plugins] Netconfig tasks changes

[Aleksandr Didenko]

Hey Adam,

in Fuel we have the following option (checkbox) on Network Setting tab:

Assign public network to all nodes
When disabled, public network will be assigned to controllers only

So if you uncheck it (by default it's unchecked) then public network and
'br-ex' will exist on controllers only. Other nodes won't even have
"Public" network on node interface configuration UI.

Regards,
Alex

On Wed, May 25, 2016 at 9:43 AM, Adam Heczko <ahec...@mirantis.com> wrote:

> Hello Alex,
> I have a question about the proposed changes.
> Is it possible to introduce new vlan and associated bridge only for
> controllers?
> I think about DMZ use case and possibility to expose public IPs/VIP and
> API endpoints on controllers on a completely separate L2 network (segment
> vlan/bridge) not present on any other nodes than controllers.
> Thanks.
>
> On Wed, May 25, 2016 at 9:28 AM, Aleksandr Didenko <adide...@mirantis.com>
> wrote:
>
>> Hi folks,
>>
>> we had to revert those changes [0] since it's impossible to propery
>> handle two different netconfig tasks for multi-role nodes. So everything
>> stays as it was before - we have single task 'netconfig' to configure
>> network for all roles and you don't need to change anything in your
>> plugins. Sorry for inconvenience.
>>
>> Our current plan for fixing network idempotency is to keep one task but
>> change 'cross-depends' parameter to yaql_exp. This will allow us to use
>> single 'netconfig' task for all roles but at the same time we'll be able to
>> properly order it: netconfig on non-controllers will be executed only
>> aftetr 'virtual_ips' task.
>>
>> Regards,
>> Alex
>>
>> [0] https://review.openstack.org/#/c/320530/
>>
>>
>> On Thu, May 19, 2016 at 2:36 PM, Aleksandr Didenko <adide...@mirantis.com
>> > wrote:
>>
>>> Hi all,
>>>
>>> please be aware that now we have two netconfig tasks (in Fuel 9.0+):
>>>
>>> - netconfig-controller - executed on controllers only
>>> - netconfig - executed on all other nodes
>>>
>>> puppet manifest is the same, but tasks are different. We had to do this
>>> [0] in order to fix network idempotency issues [1].
>>>
>>> So if you have 'netconfig' requirements in your plugin's tasks, please
>>> make sure to add 'netconfig-controller' as well, to work properly on
>>> controllers.
>>>
>>> Regards,
>>> Alex
>>>
>>> [0] https://bugs.launchpad.net/fuel/+bug/1541309
>>> [1]
>>> https://review.openstack.org/#/q/I229957b60c85ed94c2d0ba829642dd6e465e9eca,n,z
>>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Adam Heczko
> Security Engineer @ Mirantis Inc.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] [Plugins] Netconfig tasks changes

[Aleksandr Didenko]

Hi folks,

we had to revert those changes [0] since it's impossible to propery handle
two different netconfig tasks for multi-role nodes. So everything stays as
it was before - we have single task 'netconfig' to configure network for
all roles and you don't need to change anything in your plugins. Sorry for
inconvenience.

Our current plan for fixing network idempotency is to keep one task but
change 'cross-depends' parameter to yaql_exp. This will allow us to use
single 'netconfig' task for all roles but at the same time we'll be able to
properly order it: netconfig on non-controllers will be executed only
aftetr 'virtual_ips' task.

Regards,
Alex

[0] https://review.openstack.org/#/c/320530/


On Thu, May 19, 2016 at 2:36 PM, Aleksandr Didenko <adide...@mirantis.com>
wrote:

> Hi all,
>
> please be aware that now we have two netconfig tasks (in Fuel 9.0+):
>
> - netconfig-controller - executed on controllers only
> - netconfig - executed on all other nodes
>
> puppet manifest is the same, but tasks are different. We had to do this
> [0] in order to fix network idempotency issues [1].
>
> So if you have 'netconfig' requirements in your plugin's tasks, please
> make sure to add 'netconfig-controller' as well, to work properly on
> controllers.
>
> Regards,
> Alex
>
> [0] https://bugs.launchpad.net/fuel/+bug/1541309
> [1]
> https://review.openstack.org/#/q/I229957b60c85ed94c2d0ba829642dd6e465e9eca,n,z
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] YAQL console for master node

[Aleksandr Didenko]

Hi,

thank you Stas, long awaited tool :) Using it right now on the latest
Fuel-10.0, very helpful and saves a lot of time (switching between nodes to
test yaql for different roles is super cool).

Regards,
Alex


On Tue, May 24, 2016 at 12:50 PM, Stanislaw Bogatkin  wrote:

> Hi all,
>
> as you maybe know, new conditions for Fuel tasks were recently (in master
> and mitaka branches) introduced. Right after this I got several questions
> like 'hey, how can I check my new condition?' Answer could be 'use standard
> yaql console', but it hasn't have Fuel internal yaql functions which were a
> foundation for Fuel task conditions. As a result, I have written small
> utility to give an opportunity for check new conditions on the fly: [0]. It
> still in development but usable for most tasks developer usually need when
> build new yaql condition for task.
>
> If you has any questions about using this tool or want to propose any
> improvement - don't hesitate and contact me. Or just fork it and do what
> you want - it licensed under GPLv3. I would be glad if it helps someone.
>
> [0] https://github.com/sorrowless/fuyaql
>
> --
> with best regards,
> Stan.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] [Plugins] Netconfig tasks changes

[Aleksandr Didenko]

Hi all,

please be aware that now we have two netconfig tasks (in Fuel 9.0+):

- netconfig-controller - executed on controllers only
- netconfig - executed on all other nodes

puppet manifest is the same, but tasks are different. We had to do this [0]
in order to fix network idempotency issues [1].

So if you have 'netconfig' requirements in your plugin's tasks, please make
sure to add 'netconfig-controller' as well, to work properly on controllers.

Regards,
Alex

[0] https://bugs.launchpad.net/fuel/+bug/1541309
[1]
https://review.openstack.org/#/q/I229957b60c85ed94c2d0ba829642dd6e465e9eca,n,z
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][library] Update of astute.yaml fixtures and noop tests

[Aleksandr Didenko]

Alex, we can do this (and I hope we'll do) after we fix
https://bugs.launchpad.net/fuel/+bug/1567367

Regards,
Alex

On Thu, Apr 7, 2016 at 5:04 PM, Alex Schultz <aschu...@mirantis.com> wrote:

>
> On Thu, Apr 7, 2016 at 7:41 AM, Aleksandr Didenko <adide...@mirantis.com>
> wrote:
>
>> Hi,
>>
>> thanks to Dima, we now have ROLE annotations in noop tests [0]. I've
>> updated all the noop rspec tests that we currently have and added
>> appropriate role annotation [1]. So after this patch is merged, we no
>> longer need to put any new fixtures into dozens of rspec files in order to
>> enable it.
>> Please make sure to update ROLE annotations if you introduce new roles
>> (deployment groups) or change task-to-roles assignments in *tasks.yaml
>> files. Core reviewers, please don't forget to check this as well ;)
>>
>>
> Is there a reason we can't leverage the existing definitions in the
> tasks.yaml files for this?  It seems like requiring people to provide this
> information might lead to cases where it's gets out of sync. Shouldn't we
> use the task files as the source of truth for the roles?
>
> -Alex
>
>
>> Regards,
>> Alex
>>
>> [0] https://review.openstack.org/300649
>> [1] https://review.openstack.org/302313
>>
>> On Tue, Apr 5, 2016 at 12:11 PM, Aleksandr Didenko <adide...@mirantis.com
>> > wrote:
>>
>>> Hi folks,
>>>
>>> we've merged all the changes related to fixtures update [0] and bugfix
>>> to unblock noop tests [1]. So if you see -1 from fuel_noop_tests [2] in
>>> tests not related to your patch, then please rebase.
>>>
>>> Regards,
>>> Alex
>>>
>>> [0] https://review.openstack.org/#/q/topic:update-fixtures-to-9.0
>>> [1] https://review.openstack.org/301107
>>> [2] https://ci.fuel-infra.org/job/fuellib_noop_tests/
>>>
>>> On Fri, Apr 1, 2016 at 7:16 PM, Vladimir Kuklin <vkuk...@mirantis.com>
>>> wrote:
>>>
>>>> Hi Alex
>>>>
>>>> +1 to your proposal - this is long-awaited change.
>>>>
>>>> On Fri, Apr 1, 2016 at 6:01 PM, Aleksandr Didenko <
>>>> adide...@mirantis.com> wrote:
>>>>
>>>>> One more thing about spec to fixture mapping [0]. What if instead of:
>>>>>
>>>>> # RUN: (hiera1) (facts1)
>>>>>
>>>>> we'll use
>>>>>
>>>>> # RUN: (roles_array1) (facts1)
>>>>>
>>>>> ?
>>>>>
>>>>> We don't need to duplicate complicated task graph calculations to
>>>>> understand which task to execute, because we don't care about tasks
>>>>> ordering and dependencies in noop tests. All we need is to map rspec task
>>>>> tests to astute.yaml fixtures. And it could be done via roles.
>>>>>
>>>>> Regards,
>>>>> Alex
>>>>>
>>>>> [0]
>>>>> https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/usage.rst#spec-file-annotations
>>>>>
>>>>>
>>>>> On Fri, Apr 1, 2016 at 4:05 PM, Aleksandr Didenko <
>>>>> adide...@mirantis.com> wrote:
>>>>>
>>>>>> Hi.
>>>>>>
>>>>>>   As you may know, we're still using some very old astute.yaml
>>>>>> fixtures (v6.1) in our 'master' (v9.0) noop rspec tests [0]. Besides 
>>>>>> that,
>>>>>> we have problems with fixture-to-rspec mapping [1]. So we've started to
>>>>>> work on those problems [2].
>>>>>>
>>>>>>   So please be aware of upcoming changes in noop rspec fixtures and
>>>>>> tests. If you see, that some important fixtures are missing (thus not
>>>>>> covered by tests) please let me know in this email thread or via
>>>>>> IRC/email/slack.
>>>>>>
>>>>>>   Also, we should stop updating astute.yaml fixtures manually and
>>>>>> start using some kind of automation approach instead [3][4]. I propose to
>>>>>> use [5] script until we find a better solution. So if you want to add 
>>>>>> some
>>>>>> new astute.yaml fixture for noop tests, please propose a patch to this
>>>>>> script instead of uploading yaml file.
>>>>>>
>>>>>> Currently the following is mi

Re: [openstack-dev] [Fuel][library] Update of astute.yaml fixtures and noop tests

[Aleksandr Didenko]

Hi,

thanks to Dima, we now have ROLE annotations in noop tests [0]. I've
updated all the noop rspec tests that we currently have and added
appropriate role annotation [1]. So after this patch is merged, we no
longer need to put any new fixtures into dozens of rspec files in order to
enable it.
Please make sure to update ROLE annotations if you introduce new roles
(deployment groups) or change task-to-roles assignments in *tasks.yaml
files. Core reviewers, please don't forget to check this as well ;)

Regards,
Alex

[0] https://review.openstack.org/300649
[1] https://review.openstack.org/302313

On Tue, Apr 5, 2016 at 12:11 PM, Aleksandr Didenko <adide...@mirantis.com>
wrote:

> Hi folks,
>
> we've merged all the changes related to fixtures update [0] and bugfix to
> unblock noop tests [1]. So if you see -1 from fuel_noop_tests [2] in tests
> not related to your patch, then please rebase.
>
> Regards,
> Alex
>
> [0] https://review.openstack.org/#/q/topic:update-fixtures-to-9.0
> [1] https://review.openstack.org/301107
> [2] https://ci.fuel-infra.org/job/fuellib_noop_tests/
>
> On Fri, Apr 1, 2016 at 7:16 PM, Vladimir Kuklin <vkuk...@mirantis.com>
> wrote:
>
>> Hi Alex
>>
>> +1 to your proposal - this is long-awaited change.
>>
>> On Fri, Apr 1, 2016 at 6:01 PM, Aleksandr Didenko <adide...@mirantis.com>
>> wrote:
>>
>>> One more thing about spec to fixture mapping [0]. What if instead of:
>>>
>>> # RUN: (hiera1) (facts1)
>>>
>>> we'll use
>>>
>>> # RUN: (roles_array1) (facts1)
>>>
>>> ?
>>>
>>> We don't need to duplicate complicated task graph calculations to
>>> understand which task to execute, because we don't care about tasks
>>> ordering and dependencies in noop tests. All we need is to map rspec task
>>> tests to astute.yaml fixtures. And it could be done via roles.
>>>
>>> Regards,
>>> Alex
>>>
>>> [0]
>>> https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/usage.rst#spec-file-annotations
>>>
>>>
>>> On Fri, Apr 1, 2016 at 4:05 PM, Aleksandr Didenko <adide...@mirantis.com
>>> > wrote:
>>>
>>>> Hi.
>>>>
>>>>   As you may know, we're still using some very old astute.yaml fixtures
>>>> (v6.1) in our 'master' (v9.0) noop rspec tests [0]. Besides that, we have
>>>> problems with fixture-to-rspec mapping [1]. So we've started to work on
>>>> those problems [2].
>>>>
>>>>   So please be aware of upcoming changes in noop rspec fixtures and
>>>> tests. If you see, that some important fixtures are missing (thus not
>>>> covered by tests) please let me know in this email thread or via
>>>> IRC/email/slack.
>>>>
>>>>   Also, we should stop updating astute.yaml fixtures manually and start
>>>> using some kind of automation approach instead [3][4]. I propose to use [5]
>>>> script until we find a better solution. So if you want to add some new
>>>> astute.yaml fixture for noop tests, please propose a patch to this script
>>>> instead of uploading yaml file.
>>>>
>>>> Currently the following is missing in the new set of fixtures for
>>>> fuel-9.0:
>>>> - generate_vms ('vms_conf' array in astute.yaml - I'm not sure how to
>>>> properly enable it via nailgun, any help is much appreciated)
>>>> - selective ssl fixtures - since configuration data is not serialized
>>>> from nailgun, I think that we should move this into 'hiera/override' along
>>>> with implementation of new hiera overrides tests workflow [6]
>>>> - vmware related fixtures
>>>>
>>>> Please feel free to share your ideas/comments on this topic.
>>>>
>>>> Thanks,
>>>> Alex
>>>>
>>>> [0] https://bugs.launchpad.net/fuel/+bug/1535339
>>>> [1]
>>>> https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/usage.rst#spec-file-annotations
>>>> [2] https://review.openstack.org/#/q/topic:update-fixtures-to-9.0
>>>> [3]
>>>> https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/fixtures.rst
>>>> [4]
>>>> https://blueprints.launchpad.net/fuel/+spec/deployment-dryrun-fixtures-generator
>>>> [5]
>>>> https://github.com/openstack/fuel-noop-fixtures/blob/master/utils/generate_yamls.sh
>>>> [6] https://bugs.launchpad.net/fuel/+bug/1564919
>>>>
>>>
>>>
>>>
>

Re: [openstack-dev] [Fuel][library] Update of astute.yaml fixtures and noop tests

[Aleksandr Didenko]

Hi folks,

we've merged all the changes related to fixtures update [0] and bugfix to
unblock noop tests [1]. So if you see -1 from fuel_noop_tests [2] in tests
not related to your patch, then please rebase.

Regards,
Alex

[0] https://review.openstack.org/#/q/topic:update-fixtures-to-9.0
[1] https://review.openstack.org/301107
[2] https://ci.fuel-infra.org/job/fuellib_noop_tests/

On Fri, Apr 1, 2016 at 7:16 PM, Vladimir Kuklin <vkuk...@mirantis.com>
wrote:

> Hi Alex
>
> +1 to your proposal - this is long-awaited change.
>
> On Fri, Apr 1, 2016 at 6:01 PM, Aleksandr Didenko <adide...@mirantis.com>
> wrote:
>
>> One more thing about spec to fixture mapping [0]. What if instead of:
>>
>> # RUN: (hiera1) (facts1)
>>
>> we'll use
>>
>> # RUN: (roles_array1) (facts1)
>>
>> ?
>>
>> We don't need to duplicate complicated task graph calculations to
>> understand which task to execute, because we don't care about tasks
>> ordering and dependencies in noop tests. All we need is to map rspec task
>> tests to astute.yaml fixtures. And it could be done via roles.
>>
>> Regards,
>> Alex
>>
>> [0]
>> https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/usage.rst#spec-file-annotations
>>
>>
>> On Fri, Apr 1, 2016 at 4:05 PM, Aleksandr Didenko <adide...@mirantis.com>
>> wrote:
>>
>>> Hi.
>>>
>>>   As you may know, we're still using some very old astute.yaml fixtures
>>> (v6.1) in our 'master' (v9.0) noop rspec tests [0]. Besides that, we have
>>> problems with fixture-to-rspec mapping [1]. So we've started to work on
>>> those problems [2].
>>>
>>>   So please be aware of upcoming changes in noop rspec fixtures and
>>> tests. If you see, that some important fixtures are missing (thus not
>>> covered by tests) please let me know in this email thread or via
>>> IRC/email/slack.
>>>
>>>   Also, we should stop updating astute.yaml fixtures manually and start
>>> using some kind of automation approach instead [3][4]. I propose to use [5]
>>> script until we find a better solution. So if you want to add some new
>>> astute.yaml fixture for noop tests, please propose a patch to this script
>>> instead of uploading yaml file.
>>>
>>> Currently the following is missing in the new set of fixtures for
>>> fuel-9.0:
>>> - generate_vms ('vms_conf' array in astute.yaml - I'm not sure how to
>>> properly enable it via nailgun, any help is much appreciated)
>>> - selective ssl fixtures - since configuration data is not serialized
>>> from nailgun, I think that we should move this into 'hiera/override' along
>>> with implementation of new hiera overrides tests workflow [6]
>>> - vmware related fixtures
>>>
>>> Please feel free to share your ideas/comments on this topic.
>>>
>>> Thanks,
>>> Alex
>>>
>>> [0] https://bugs.launchpad.net/fuel/+bug/1535339
>>> [1]
>>> https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/usage.rst#spec-file-annotations
>>> [2] https://review.openstack.org/#/q/topic:update-fixtures-to-9.0
>>> [3]
>>> https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/fixtures.rst
>>> [4]
>>> https://blueprints.launchpad.net/fuel/+spec/deployment-dryrun-fixtures-generator
>>> [5]
>>> https://github.com/openstack/fuel-noop-fixtures/blob/master/utils/generate_yamls.sh
>>> [6] https://bugs.launchpad.net/fuel/+bug/1564919
>>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com <http://www.mirantis.ru/>
> www.mirantis.ru
> vkuk...@mirantis.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][library] Update of astute.yaml fixtures and noop tests

[Aleksandr Didenko]

One more thing about spec to fixture mapping [0]. What if instead of:

# RUN: (hiera1) (facts1)

we'll use

# RUN: (roles_array1) (facts1)

?

We don't need to duplicate complicated task graph calculations to
understand which task to execute, because we don't care about tasks
ordering and dependencies in noop tests. All we need is to map rspec task
tests to astute.yaml fixtures. And it could be done via roles.

Regards,
Alex

[0]
https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/usage.rst#spec-file-annotations


On Fri, Apr 1, 2016 at 4:05 PM, Aleksandr Didenko <adide...@mirantis.com>
wrote:

> Hi.
>
>   As you may know, we're still using some very old astute.yaml fixtures
> (v6.1) in our 'master' (v9.0) noop rspec tests [0]. Besides that, we have
> problems with fixture-to-rspec mapping [1]. So we've started to work on
> those problems [2].
>
>   So please be aware of upcoming changes in noop rspec fixtures and tests.
> If you see, that some important fixtures are missing (thus not covered by
> tests) please let me know in this email thread or via IRC/email/slack.
>
>   Also, we should stop updating astute.yaml fixtures manually and start
> using some kind of automation approach instead [3][4]. I propose to use [5]
> script until we find a better solution. So if you want to add some new
> astute.yaml fixture for noop tests, please propose a patch to this script
> instead of uploading yaml file.
>
> Currently the following is missing in the new set of fixtures for fuel-9.0:
> - generate_vms ('vms_conf' array in astute.yaml - I'm not sure how to
> properly enable it via nailgun, any help is much appreciated)
> - selective ssl fixtures - since configuration data is not serialized from
> nailgun, I think that we should move this into 'hiera/override' along with
> implementation of new hiera overrides tests workflow [6]
> - vmware related fixtures
>
> Please feel free to share your ideas/comments on this topic.
>
> Thanks,
> Alex
>
> [0] https://bugs.launchpad.net/fuel/+bug/1535339
> [1]
> https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/usage.rst#spec-file-annotations
> [2] https://review.openstack.org/#/q/topic:update-fixtures-to-9.0
> [3]
> https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/fixtures.rst
> [4]
> https://blueprints.launchpad.net/fuel/+spec/deployment-dryrun-fixtures-generator
> [5]
> https://github.com/openstack/fuel-noop-fixtures/blob/master/utils/generate_yamls.sh
> [6] https://bugs.launchpad.net/fuel/+bug/1564919
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel][library] Update of astute.yaml fixtures and noop tests

[Aleksandr Didenko]

Hi.

  As you may know, we're still using some very old astute.yaml fixtures
(v6.1) in our 'master' (v9.0) noop rspec tests [0]. Besides that, we have
problems with fixture-to-rspec mapping [1]. So we've started to work on
those problems [2].

  So please be aware of upcoming changes in noop rspec fixtures and tests.
If you see, that some important fixtures are missing (thus not covered by
tests) please let me know in this email thread or via IRC/email/slack.

  Also, we should stop updating astute.yaml fixtures manually and start
using some kind of automation approach instead [3][4]. I propose to use [5]
script until we find a better solution. So if you want to add some new
astute.yaml fixture for noop tests, please propose a patch to this script
instead of uploading yaml file.

Currently the following is missing in the new set of fixtures for fuel-9.0:
- generate_vms ('vms_conf' array in astute.yaml - I'm not sure how to
properly enable it via nailgun, any help is much appreciated)
- selective ssl fixtures - since configuration data is not serialized from
nailgun, I think that we should move this into 'hiera/override' along with
implementation of new hiera overrides tests workflow [6]
- vmware related fixtures

Please feel free to share your ideas/comments on this topic.

Thanks,
Alex

[0] https://bugs.launchpad.net/fuel/+bug/1535339
[1]
https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/usage.rst#spec-file-annotations
[2] https://review.openstack.org/#/q/topic:update-fixtures-to-9.0
[3]
https://github.com/openstack/fuel-noop-fixtures/blob/master/doc/fixtures.rst
[4]
https://blueprints.launchpad.net/fuel/+spec/deployment-dryrun-fixtures-generator
[5]
https://github.com/openstack/fuel-noop-fixtures/blob/master/utils/generate_yamls.sh
[6] https://bugs.launchpad.net/fuel/+bug/1564919
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] [FFE] FF exception request for SR-IOV

[Aleksandr Didenko]

Hi,

feature has been merged. QA is in progress.

Thanks,
Alex

On Fri, Mar 4, 2016 at 1:15 AM, Dmitry Borodaenko <dborodae...@mirantis.com>
wrote:

> Granted, merge deadline March 16, feature to be marked experimental
> until QA has signed off that it's fully tested and stable.
>
> --
> Dmitry Borodaenko
>
>
> On Wed, Mar 02, 2016 at 01:40:26PM +0200, Aleksey Kasatkin wrote:
> > > And we need to write a new patch for:
> >
> https://blueprints.launchpad.net/fuel/+spec/nailgun-should-serialize-sriov
> >
> > It is on review now: https://review.openstack.org/286704
> >
> > Complete list of patches on review:
> >
> https://review.openstack.org/#/q/status:open++branch:master+topic:bp/support-sriov
> >
> >
> >
> > Aleksey Kasatkin
> >
> >
> > On Tue, Mar 1, 2016 at 6:27 PM, Aleksandr Didenko <adide...@mirantis.com
> >
> > wrote:
> >
> > > Hi,
> > >
> > > I'd like to to request a feature freeze exception for "Support for
> SR-IOV
> > > for improved networking performance" feature [0].
> > >
> > > Part of this feature is already merged [1]. We have the following
> patches
> > > in work / on review:
> > >
> > > https://review.openstack.org/280782
> > > https://review.openstack.org/284603
> > > https://review.openstack.org/286633
> > >
> > > And we need to write a new patch for:
> > >
> https://blueprints.launchpad.net/fuel/+spec/nailgun-should-serialize-sriov
> > >
> > > We need 2 weeks at most after FF to accomplish this.
> > > Risk of not delivering it after 2 weeks is very low.
> > >
> > > Regards,
> > > Alex
> > >
> > > [0] https://blueprints.launchpad.net/fuel/+spec/support-sriov
> > > [1]
> > >
> https://review.openstack.org/#/q/status:merged+branch:master+topic:bp/support-sriov
> > >
> > >
> > >
> __
> > > OpenStack Development Mailing List (not for usage questions)
> > > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
> > >
>
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] network_metadata hash keys on astute.yaml

[Aleksandr Didenko]

Hi,

some additional info on the problem: if I create some Hiera override for
the nodes list and use node key which is hostname bond, then after node
rename (rename during LCM or reset/rename/redeploy - doesn't matter) my
override will create a "ghost" node in the list and will not change
settings I wanted to change. So node keys in that hash should remain
immutable.

Let's fix LP#1538220 and keep 'node-{uid}' simply because that's how it was
working before (and does not require new patches like [0]) and we're too
late in the release cycle to change keys to '{uid}'.

Regards,
Alex

[0] https://review.openstack.org/#/c/284046/

On Tue, Mar 15, 2016 at 11:22 AM, Kyrylo Galanov 
wrote:

> Hi,
>
> Currently nailgun and puppet process network_metadata hash slightly
> different.
> Nailgun uses short hostname as a hash key for each node, while library
> code assumes that key is always 'node-{uid}'. Sometimes it can result in a
> deployment failure.
>
> During code review[0] it turned out that there are two points of view,
> which approach is correct [1].
> Both are one-line fixes, however it have been lasting too long with no
> result.
>
> I would like to start a discussion with a hope to close the issue shortly.
>
> Best regards.
> Kyrylo
>
> [0] https://review.openstack.org/#/c/284046/
> [1] https://bugs.launchpad.net/fuel/+bug/1538220
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Compatibility of fuel plugins and fuel versions

[Aleksandr Didenko]

> Good idea. That should be done despite on any decision we will take.

Currently you have to specify which releases your plugin supports [0]. So I
can take my plugin developed for 8.0 and install it on Fuel-9.0 (I actually
did it and it worked just fine). But I won't be able to enable this plugin
for "mitaka-9.0" release because plugin was not developed and tested for
it, so it does not have "miraka-9.0" in "releases" list [0]. So I don't
quite understand how new "validated_against" parameter will differ from
existing "releases" list.

Regards,
Alex

[0]
https://github.com/openstack/fuel-plugin-external-lb/blob/68fc91a2d3360f19605180d7c3d8683227c8d5b1/metadata.yaml#L11-L21


On Thu, Mar 10, 2016 at 10:22 AM, Bogdan Dobrelya 
wrote:

> On 10.03.2016 08:30, Mike Scherbakov wrote:
> > Hi folks,
> > in order to make a decision whether we need to support example plugins,
> > and if actually need them [1], I'd suggest to discuss more common things
> > about plugins.
> >
> > My thoughts:
> > 1) This is not good, that our plugins created for Fuel 8 won't even
> > install on Fuel 9. By default, we should assume that plugin will work at
> > newer version of Fuel. However, for proper user experience, I suggest to
> > create meta-field "validated_against", where plugin dev would provide
> > versions of Fuel this plugin has been tested with. Let's say, it was
> > tested against 7.0, 8.0. If user installs plugin in Fuel 9, I'd suggest
> > to show a warning saying about risks and the fact that the plugin has
> > not been tested against 9. We should not restrict intsallation against
> > 9, though.
>
> Good idea. That should be done despite on any decision we will take.
>
> >
> > 2) We need to keep backward compatibility of pluggable interface for a
> > few releases. So that plugin developer can use pluggable interface of
> > version x, which was supported in Fuel 6.1. If we still support it, it
> > would mean (see next point) compatibility of this plugin with 6.1, 7.0,
> > 8.0, 9.0. If we want to deprecate pluggable interface version, we should
> > announce it, and basically follow standard process of deprecation.
> >
> > 3) Plugin's ability to work against multiple releases of Fuel
> > (multi-release support). If if..else clauses to support multiple
> > releases are fairly minimal, let's say take less that 10% of LOC, I'd
> > suggest to have this supported. Just because it will be easier for
> > plugin devs to support their plugin code (no code duplication, single
> > repo for multiple releases).
> >
> > Thoughts?
> >
> > [1]
> http://lists.openstack.org/pipermail/openstack-dev/2016-March/088211.html
> > --
> > Mike Scherbakov
> > #mihgen
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Overriding and removing VIPs from plugins

[Aleksandr Didenko]

Guys,

I suppose that any solution for the plugins merge ordering issue (either
alerting about error or allowing user to manually set order) would be very
helpful and much better then no solution at all (like we currently have).
Maybe we should create a blueprint to not miss/forget about it?

Regards,
Alex

On Mon, Feb 1, 2016 at 6:10 PM, Roman Prykhodchenko <m...@romcheg.me> wrote:

> This is basically why I proposed to allow users to set per cluster
> priority for every enabled plugin. That would allow users to select results
> they need. Putting too many logic into checks whether two plugins are
> incompatible is error-prone and won’t stop anyone from shooting their feet.
>
>
> > 29 січ. 2016 р. о 16:10 Igor Kalnitsky <ikalnit...@mirantis.com>
> написав(ла):
> >
> > Roman P. wrote:
> >> Putting extra checks will create a more fool-proof but less configurable
> >> software. I’d vote for letting users shoot their feet using their
> plugins
> >> but making Fuel more flexible.
> >
> > I won't agree here. You see, what if two plugins wants to override the
> > core network role? Consider that plugin A wants to extend VIPs:
> >
> >id: "mgmt/vip"
> >default_mapping: "management"
> >properties:
> >  vip:
> >- name: "management"
> >  namespace: "haproxy"
> ># new VIP we want to add
> >- name: "myvip"
> >  namespace: "haproxy"
> >
> > while plugin B wants to remove them:
> >
> >id: "mgmt/vip"
> >default_mapping: "management"
> >properties:
> >  vip: []
> >
> > How do you suppose to resolve this action? We don't know in which
> > order they will be resolved, and I'm strongly against unpredictable
> > situation (especiall it may be different time-to-time and depends on
> > which order plugins will be selected).
> >
> > Moreover, it makes a little sense to try to resolve this situation. If
> > two plugins change something in core, they are probably incompatible.
> > Manual actions are required.
> >
> > So that's, basically, why I think we should warn user about
> > incompatible changes to core stuff. Just like we do for deployment
> > tasks.
> >
> > - igor
> >
> > On Fri, Jan 29, 2016 at 4:18 PM, Roman Prykhodchenko <m...@romcheg.me>
> wrote:
> >> I would not check that. We are not talking about installing browser
> plugins
> >> when users may not know what they want. Putting extra checks will
> create a
> >> more fool-proof but less configurable software. I’d vote for letting
> users
> >> shoot their feet using their plugins but making Fuel more flexible.
> >>
> >>
> >> 29 січ. 2016 р. о 15:05 Aleksey Kasatkin <akasat...@mirantis.com>
> >> написав(ла):
> >>
> >>> jsonpatch
> >>
> >> There were +1's regarding overriding VIPs above. I'd stick with that.
> It is
> >> done for tasks now. But we will need to check conflicts between plugins
> >> there (as for tasks).
> >>
> >>
> >> Aleksey Kasatkin
> >>
> >>
> >> On Fri, Jan 29, 2016 at 1:23 PM, Roman Prykhodchenko <m...@romcheg.me>
> wrote:
> >>>
> >>> Frankly, I have not though about how to deal with multiple plugins yet.
> >>> However, what I think is that we must not restrict this too much and
> let
> >>> users configure it more flexibly even if it allows to shoot one’s foot.
> >>> Perhaps we can add a per-cluster priority for enabled plugins which
> users
> >>> can configure via UI, CLI or API. My other thought is that we should
> not
> >>> invent our own mechanics for changing a configuration and use an
> existing
> >>> one, say, jsonpatch. What do you guys think?
> >>>
> >>> P. S. [0] is really needed for 8.0 for implementing an important epic,
> so
> >>> please check it out. If it does not break anything, lets merge it ASAP.
> >>>
> >>> - romcheg
> >>>
> >>> 28 січ. 2016 р. о 18:30 Aleksey Kasatkin <akasat...@mirantis.com>
> >>> написав(ла):
> >>>
> >>> AFAIC, it is better to remove by name then. Otherwise, you do not know
> >>> what VIPs you are removing.
> >>> Another option - remove "built-in" VIPs using some specific expression.
> >>> Anyway, you do not know where other VIPs (VIPs of other plugins) live
>

Re: [openstack-dev] [Fuel] Nominate Maksim Malchuk for the fuel-virtualbox-core team

[Aleksandr Didenko]

+1

On Thu, Mar 3, 2016 at 11:00 AM, Kyrylo Galanov 
wrote:

> +1
>
> On Thu, Mar 3, 2016 at 2:53 AM, Roman Vyalov  wrote:
>
>> +1
>>
>> On Wed, Mar 2, 2016 at 5:47 PM, Sergey Kulanov 
>> wrote:
>>
>>> Hey Fuelers,
>>>
>>> Since we've successfully moved [1] virtual-box scripts from fuel-main
>>> [2] to
>>> separate fuel-virtualbox [3] git repo, I propose to update
>>> fuel-virtualbox-core
>>> team [4] by adding Maksim Malchuk. Maksim is the main contributor to
>>> these
>>> scripts during Mitaka release cycle [5]
>>>
>>> Fuel Cores, please vote.
>>>
>>> [1].
>>> http://lists.openstack.org/pipermail/openstack-dev/2016-February/086560.html
>>> [2]. https://github.com/openstack/fuel-main
>>> [3]. https://github.com/openstack/fuel-virtualbox
>>> [4]. https://review.openstack.org/#/admin/groups/1299,members
>>> [5]. https://github.com/openstack/fuel-virtualbox/commits/master
>>>
>>> --
>>> Sergey
>>> DevOps Engineer
>>> IRC: SergK
>>> Skype: Sergey_kul
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Fuel-Library] Nominating Matthew Mosesohn for Fuel Library Core

[Aleksandr Didenko]

+1 for Michael Polenchuk

On Wed, Mar 2, 2016 at 5:33 PM, Fedor Zhadaev  wrote:

> +1 for Michael :)
>
> ср, 2 мар 2016, 17:50 Matthew Mosesohn :
>
>> Hi all,
>>
>> Thank you for the nominations and +1s. I would like to propose Michael
>> Polenchuk to add as a maintainer to fuel-library to take my spot when
>> I leave the maintainers list.
>>
>> Best Regards,
>> Matthew Mosesohn
>>
>> On Fri, Feb 26, 2016 at 3:54 PM, Kyrylo Galanov 
>> wrote:
>> > Finally! +2 !
>> >
>> > On Fri, Feb 26, 2016 at 9:08 PM, Roman Vyalov 
>> wrote:
>> >>
>> >> +1
>> >>
>> >> On Fri, Feb 26, 2016 at 12:31 PM, Aleksey Kasatkin
>> >>  wrote:
>> >>>
>> >>> +1
>> >>>
>> >>>
>> >>> Aleksey Kasatkin
>> >>>
>> >>>
>> >>> On Thu, Feb 25, 2016 at 11:59 PM, Sergey Vasilenko
>> >>>  wrote:
>> 
>>  +1
>> 
>> 
>>  /sv
>> 
>> 
>> 
>> 
>> __
>>  OpenStack Development Mailing List (not for usage questions)
>>  Unsubscribe:
>>  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> >>>
>> >>>
>> >>>
>> >>>
>> __
>> >>> OpenStack Development Mailing List (not for usage questions)
>> >>> Unsubscribe:
>> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>
>> >>
>> >>
>> >>
>> __
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> --
> Kind Regards,
> Fedor Zhadaev
>
> skype: zhadaevfm
> IRC: fzhadaev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] FFE request for osnailyfacter refactoring for Puppet Master compatibility

[Aleksandr Didenko]

Hi,

> Merging this code is relatively non-intrusive to core Fuel Library code
> as it is merely re-organizing the file structure of the osnailyfacter
> module to be compatible with Puppet Master.

It looks like super-intrusive to me. Modular manifests are, actually, the
core of Fuel Library. And the majority of changes we introduce in Fuel
Library are proposed for those manifests. So if you're going to move those
manifests into "osnailyfacter::*" classes then it will basically conflict
with the 90% of other patches for Fuel Library. This may slow down
development of other features as well as bug fixing.

Also I see no patches on review and spec is not yet accepted. I think
starting such an intrusive feature after FF is too risky, let's move it to
10.0.

Regards,
Alex

On Wed, Mar 2, 2016 at 1:21 AM, Scott Brimhall 
wrote:

> Greetings,
>
> As you might know, we are working on integrating a 3rd party
> configuration management platform (Puppet Master) with Fuel.
> This integration will provide the capability for state enforcement
> and will further enable "day 2" operations of a Fuel-deployed site.
> We must refactor the 'osnailyfacter' module in Fuel Library to be
> compatible with both a masterful and masterless Puppet approach.
>
> This change is required to enable a Puppet Master based LCM
> solution.
>
> We request a FFE for this feature for 3 weeks, until Mar 24.  By that
> time, we will provide a tested solution in accordance with the following
> specifications [1].
>
> The feature includes the following components:
>
> 1. Refactor 'osnailyfacter' Fuel Library module to be compatible with
> Puppet Master by becoming a valid and compliant Puppet module.
> This involves moving manifests into the proper manifests directory
> and moving the contents into classes that can be included by Puppet
> Master.
> 2. Update deployment tasks to update their manifest path to the new
> location.
>
> Merging this code is relatively non-intrusive to core Fuel Library code
> as it is merely re-organizing the file structure of the osnailyfacter
> module to be compatible with Puppet Master.  Upon updating the
> deployment tasks to reflect the new location of manifests, this feature
> remains compatible with the masterless puppet apply approach that
> Fuel uses while providing the ability to integrate a Puppet Master
> based LCM solution.
>
> Overall, I consider this change as low risk for integrity and timeline of
> the release and it is a critical feature for the ability to integrate an
> LCM
> solution using Puppet Master.
>
> Please consider our request and share concerns so we can properly
> resolve them.
>
> [1]
> https://blueprints.launchpad.net/fuel/+spec/fuel-refactor-osnailyfacter-for-puppet-master-compatibility
>
> ---
> Best Regards,
>
> Scott Brimhall
> Systems Architect
> Mirantis Inc
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] [FFE] FF exception request for DPDK

[Aleksandr Didenko]

Hi,

I'd like to to request a feature freeze exception for "Support for DPDK for
improved networking performance" feature [0].

Part of this feature is already merged [1]. We have the following patches
in work / on review:

https://review.openstack.org/281827
https://review.openstack.org/283044
https://review.openstack.org/286595
https://review.openstack.org/284285
https://review.openstack.org/284283
https://review.openstack.org/286611

And we need to write new patches for the following parts of this feature:
https://blueprints.launchpad.net/fuel/+spec/network-verification-dpdk
https://blueprints.launchpad.net/fuel/+spec/support-dpdk-bond

We need 3 weeks after FF to finish this feature.
Risk of not delivering it after 3 weeks is low.

Regards,
Alex

[0] https://blueprints.launchpad.net/fuel/+spec/support-dpdk
[1]
https://review.openstack.org/#/q/status:merged+branch:master+topic:bp/support-dpdk
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] [FFE] FF exception request for SR-IOV

[Aleksandr Didenko]

Hi,

I'd like to to request a feature freeze exception for "Support for SR-IOV
for improved networking performance" feature [0].

Part of this feature is already merged [1]. We have the following patches
in work / on review:

https://review.openstack.org/280782
https://review.openstack.org/284603
https://review.openstack.org/286633

And we need to write a new patch for:
https://blueprints.launchpad.net/fuel/+spec/nailgun-should-serialize-sriov

We need 2 weeks at most after FF to accomplish this.
Risk of not delivering it after 2 weeks is very low.

Regards,
Alex

[0] https://blueprints.launchpad.net/fuel/+spec/support-sriov
[1]
https://review.openstack.org/#/q/status:merged+branch:master+topic:bp/support-sriov
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Fuel-Library] Nominating Matthew Mosesohn for Fuel Library Core

[Aleksandr Didenko]

+1

On Wed, Feb 24, 2016 at 1:50 PM, Vladimir Kuklin 
wrote:

> Fellow Fuelers
>
> I would like to kindly ask you to consider voting for Matthew Mosesohn as
> a Fuel Library Core
> reviewer.
>
> Matthew has been working with Fuel since its inception, worked on
> countless amount of features, such as :
>
> Master Node Upgrades and Backup
> Improvements to Fuel Infra
> Mitaka, Kilo and Juno Support
> Detachable Services Plugins
> RHOS Support in Fuel
> UCA Support
> Refactoring of Haproxy and Firewall pieces
>
> He has also been known as our Fuel Master node and Docker guru and has
> been continuously working on bugfixing where he scores as a bug squashing
> monster with more than 150 bugfixes to all the parts of Fuel Library (#1
> throughout FL history).
>
> As you can see, there is not a piece of Fuel Library that he has not yet
> gained experience with.
>
> And this can easily be fetched with simple statistics of Matt's
> contribution. He is the topmost contributor to all Fuel projects among all
> Fuel Library folks and is the 3rd contributor of Fuel Library.
> He also reviews a lot and has a fair amount of -1's (he is not as cruel as
> me, though :-))
>
> Having that said, I would like to open the vote to promote Matt to
> OpenStack Fuel Library core reviewers.
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com 
> www.mirantis.ru
> vkuk...@mirantis.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Wildcards instead of

[Aleksandr Didenko]

> I vote to abandon it. Let's do not break existing plugins, and do not
> add *undo* tasks for plugin developers. If they want to configure
> network, they'll ask it explicitly.

+1 to this gentleman. It's safe to add wildcards only to tasks that were
moved from pre/post deployment stages, which were executed everywhere
anyway.

Regards,
Alex

On Fri, Feb 19, 2016 at 3:22 PM, Bulat Gaifullin 
wrote:

>
> > On 19 Feb 2016, at 17:09, Igor Kalnitsky 
> wrote:
> >
> > Kyrylo G. wrote:
> >> So who is voting for the path to be abandoned?
> >
> > I vote to abandon it. Let's do not break existing plugins, and do not
> > add *undo* tasks for plugin developers. If they want to configure
> > network, they'll ask it explicitly.
> >
> >
> > Kyrylo G. wrote:
> >> By the way, there is already a task running by the wildcard:
> >>
> https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/fuel_pkgs/tasks.yaml#L4
> >
> > Yes, exactly, but the thing is that our original task for setuping
> > repos was executed on all nodes before, including ones provided by
> > plugins. Making it executing on core nodes only may break plugins that
> > rely on it. So generally, it's about backward compatibility.
> >
> >
> > Bulat G. wrote:
> >> This tasks should run on all nodes and it does not matter, the node
> >> has role from plugin or core-role.
> >
> > Nope, they shouldn't. Why do I need to install the following packages
> >
> >  'screen',
> >  'tmux',
> >  'htop',
> >  'tcpdump',
> >  'strace',
> >  'fuel-misc',
> >  'man-db',
> >  'fuel-misc',
> >  'fuel-ha’
> >
> It is big problem?
>
> > if I have no plans to use them? As a deployer engineer, I'd prefer to
> > keep my role as clear as possible, and decide what to install in my
> > own way.
>
> IMO: The plugin developer wants to install additional applications to
> extend functionality, It do not want configure low-level things, like
> specify some banch of task for configure network, configure repositories
> etc.
> How can we manage new node if network is not configured or fuel-agent is
> not installed?
>
> >
> >
> > On Fri, Feb 19, 2016 at 1:06 PM, Bulat Gaifullin
> >  wrote:
> >> +1 to use wildcards for common tasks like netconfig and setup
> repositories.
> >> This tasks should run on all nodes and it does not matter, the node has
> role
> >> from plugin or core-role.
> >> In my opinion we should one approach for basic configuration of node.
> >>
> >> Regards,
> >> Bulat Gaifullin
> >> Mirantis Inc.
> >>
> >>
> >>
> >> On 19 Feb 2016, at 13:36, Kyrylo Galanov  wrote:
> >>
> >> Hi,
> >>
> >> So who is voting for the path to be abandoned?
> >>
> >> By the way, there is already a task running by the wildcard:
> >>
> https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/fuel_pkgs/tasks.yaml#L4
> >> However, it this case it might work with plugins.
> >>
> >> Best regards,
> >> Kyrylo
> >>
> >> On Fri, Feb 19, 2016 at 1:09 AM, Igor Kalnitsky <
> ikalnit...@mirantis.com>
> >> wrote:
> >>>
> >>> Hey Kyrylo,
> >>>
> >>> As it was mentioned in the review: you're about to break roles defined
> >>> by plugins. That's not good move, I believe.
> >>>
> >>> Regarding 'exclude' directive, I have no idea what you're talking
> >>> about. We don't support it now, and, anyway, there should be no
> >>> difference between roles defined by plugins and core roles.
> >>>
> >>> - Igor
> >>>
> >>> On Thu, Feb 18, 2016 at 12:53 PM, Kyrylo Galanov <
> kgala...@mirantis.com>
> >>> wrote:
>  Hello,
> 
>  We are about to switch to wildcards instead of listing all groups in
>  tasks
>  explicitly [0].
>  This change must make deployment process more obvious for developers.
>  However, it might lead to confusion when new groups are added either
> by
>  plugin or fuel team in future.
> 
>  As mention by Bogdan, it is possible to use 'exclude' directive to
>  mitigate
>  the risk.
>  Any thoughts on the topic are appreciated.
> 
> 
>  [0] https://review.openstack.org/#/c/273596/
> 
>  Best regards,
>  Kyrylo
> 
> 
> 
> __
>  OpenStack Development Mailing List (not for usage questions)
>  Unsubscribe:
>  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> >>>
> >>>
> __
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> >>
> __
> >> OpenStack Development Mailing 

Re: [openstack-dev] [fuel] Supporting multiple Openstack versions

[Aleksandr Didenko]

> Given the requirements to be able to use new features in fuel, with an
older version of OpenStack, what alternative would you propose?

For example, it's possible to use existing "release" functionality in Fuel
(release contains granular tasks configuration, puppet modules and
manifests, configuration data). So after upgrade from 8.0 to 9.0 it will
look like this [0] - with separate composition layer for every supported
"release".

> We should allow a user to specify that they want a build a cloud using X
fuel release to deploy Y os with Z OpenStack release.

[0] should work for this as well. But the number of X-Y-Z combinations will
be limited. Well, it will be limited in any case, I don't think that it's
possible to support unlimited number of OpenStack versions in a single Fuel
release.

In case we want to use single composition layer for more than one openstack
version, we need to resolve the following blockers:
- Move everything except composition layer (top-scope manifests and other
granular tasks) from fuel-library to their own repos. Otherwise we'll have
OpenStack version conditionals in modules manifets, providers and functions
which would be a mess.
- Refactor tasks upload/serialization in Nailgun
- (?) Refactor configuration data serialization in Nailgun

And still we'll have to add conditionals to puppet functions that relay on
configuration data directly (like generate_network_config.rb). Or write
some sort of data serialization in front of them in manifests. Or leave
nailgun serialization based on installed version (which is almost the same
as using separate composition layers [0]).

In either case (separate releases or single composition layer) it will
double CI load and testing efforts, because we need to CI/test new features
and patches for 9.0+mitaka and 9.0+liberty.

Regards,
Alex

[0] http://paste.openstack.org/show/487383/


On Thu, Feb 18, 2016 at 9:31 AM, Bogdan Dobrelya 
wrote:

> On 17.02.2016 18:23, Bogdan Dobrelya wrote:
> >> So we'll have tons of conditionals in composition layer, right? Even if
> >> some puppet-openstack class have just one new parameter in new release,
> >> then we'll have to write a conditional and duplicate class declaration.
> Or
> >> write complex parameters hash definitions/merges and use
> >> create_resources(). The more releases we want to support the more
> >> complicated composition layer will become. That won't make contribution
> to
> >> fuel-library easier and even can greatly reduce development speed. Also
> are
> >> we going to add new features to stable releases using this workflow with
> >> single composition layer?
> >
> > As I can see from an example composition [0], such code would be an
> > unmaintainable burden for development and QA process. Next imagine a
> > case for incompatible *providers* like network transformations - shall
> > we put multiple if/case to the ruby providers as well?..
> >
> > That is not a way to go for a composition, sorry. While the idea may be
> > doable, I agree, but perhaps another way.
> >
> > (tl;dr)
> > By the way, this reminded me "The wrong abstraction" [1] article and
> > discussion. I agree with the author and believe one should not group
> > code (here it is versioned puppet modules & compositions) in a way which
> > introduces abstractions (here a super-composition) with multiple
> > if/else/case and hardcoded things to switch the execution flow based on
> > version of things. Just keep code as is - partially duplicated by
> > different releases in separate directories with separate modules and
> > composition layers and think of better solutions please.
> >
> > There is also a nice comment: "...try to optimize my code around
> > reducing state, coupling, complexity and code, in that order". I
> > understood that like a set of "golden rules":
> > - Make it coupled more tight to decrease (shared) state
> > - Make it more complex to decrease coupling
> > - Make it duplicated to decrease complexity (e.g. abstractions)
> >
> > (tl;dr, I mean it)
> > So, bringing those here.
> > - The shared state is perhaps the Nailgun's world view of all data and
> > versioned serializers for supported releases, which know how to convert
> > the only latest existing data to any of its supported previous versions.
> > - Decoupling we do by putting modules with its compositions to different
> > versioned /etc/puppet subdirectories. I'm not sure how do we decouple
> > Nailgun serializers though.
> > - Complexity is how we compose those modules / write logic of
> serializers.
> > - Duplication is puppet classes (and providers) with slightly different
> > call parameters from a version to version. Sometimes even not backwards
> > compatible. Probably same to the serializers?
> >
> > So, we're going to *increase complexity* by introducing
> > super-compositions for multi OpenStack releases. Not sure about what to
> > happen to the serializers, any volunteers to clarify an impact?. And the
> > Rules 

Re: [openstack-dev] [fuel] Supporting multiple Openstack versions

[Aleksandr Didenko]

> This requires the loss of all of the features in the newer version of
fuel since it relies on the older version of the serialized data from
nailgun.

Yes. But isn't it how "stable" branches are supposed to work? Introducing
new features into "stable" branches will make them not so "stable", right?
Even if these new features are introduced in composition layer or
configuration data. just an example: network transformations in astute.yaml
that are being translated into actual network configuration.

> Yes, this is, in part,  about taking advantage of new fuel features on
stable openstack releases, we are almost always behind and the previous
release(s) supported this already.

Introducing new features to stable releases will require full cycle of
testing. So, basically, it will affect the whole development process.

> In addtion we currently don't allow for new clusters to be deployed this
way.

We can remove this restriction. Nailgun is able to serialize data for
previous releases because that's how it supports adding new nodes to older
environments after upgrade, so it should not be a problem.

Regards,
Alex

On Fri, Feb 12, 2016 at 10:19 PM, Andrew Woodward <xar...@gmail.com> wrote:

>
>
> On Thu, Feb 11, 2016 at 1:03 AM Aleksandr Didenko <adide...@mirantis.com>
> wrote:
>
>> Hi,
>>
>>
>> > So what is open? The composition layer.
>>
>> We can have different composition layers for every release and it's
>> already implemented in releases - separate puppet modules/manifests dir for
>> every release.
>>
>
> This requires the loss of all of the features in the newer version of fuel
> since it relies on the older version of the serialized data from nailgun.
> In addtion we currently don't allow for new clusters to be deployed this
> way.
>
>
>>
>> > Currently, we just abandon support for previous versions in the
>> composition layer and leave them to only be monuments in the
>> stable/ series branches for maintenance. If we instead started
>> making changes (forwards or backwards that) change the calls based on the
>> openstack version [5] then we would be able to change the calls based on
>> then needs of that release, and the puppet-openstack modules we are working
>> with.
>>
>> So we'll have tons of conditionals in composition layer, right? Even if
>> some puppet-openstack class have just one new parameter in new release,
>> then we'll have to write a conditional and duplicate class declaration. Or
>> write complex parameters hash definitions/merges and use
>> create_resources(). The more releases we want to support the more
>> complicated composition layer will become. That won't make contribution to
>> fuel-library easier and even can greatly reduce development speed. Also are
>> we going to add new features to stable releases using this workflow with
>> single composition layer?
>>
>> Yes, we need conditionals in the composition layer, we already need these
> to no jam the gate when we switch between stable and master, we might as
> well maintain them properly so that we can start running multiple versions
>
> Yes, this is, in part,  about taking advantage of new fuel features on
> stable openstack releases, we are almost always behind and the previous
> release(s) supported this already.
>
> If its only supported in the newer version, then we would have a similar
> problem with enabling the feature anyways as our current process results in
> us developing on stable openstack with the newer fuel until late in the
> cycle, when we switch packaging over.
>
>>
>> > Testing master while keeping stable. Given the ability to conditional
>> what source of openstack bits, which versions of manifests we can start
>> testing both master and keep health on stable. This would help accelerate
>> both fuel development and deploying and testing development versions of
>> openstack
>>
>> I'm sorry, but I don't see how we can accelerate things by making
>> composition layer more and more complicated. If we're going to run CI and
>> swarm for all of the supported releases on the ISO, that would rather
>> decrease speed of development and testing drastically. Also aren't we
>> "testing both master and keep health on stable" right now by running tests
>> for master and stable versions of Fuel?
>>
>> No, this is about deploying stable and master from the same version of
> Fuel, with the new features from fuel. As we develop new features in fuel
> we frequently run into problems simply because openstack version we are
> deploying is broken, this would allow for gating on stable and edge testing
> master unt

Re: [openstack-dev] [fuel] Supporting multiple Openstack versions

[Aleksandr Didenko]

Hi,

> So what is open? The composition layer.

We can have different composition layers for every release and it's already
implemented in releases - separate puppet modules/manifests dir for every
release.

> Currently, we just abandon support for previous versions in the
composition layer and leave them to only be monuments in the
stable/ series branches for maintenance. If we instead started
making changes (forwards or backwards that) change the calls based on the
openstack version [5] then we would be able to change the calls based on
then needs of that release, and the puppet-openstack modules we are working
with.

So we'll have tons of conditionals in composition layer, right? Even if
some puppet-openstack class have just one new parameter in new release,
then we'll have to write a conditional and duplicate class declaration. Or
write complex parameters hash definitions/merges and use
create_resources(). The more releases we want to support the more
complicated composition layer will become. That won't make contribution to
fuel-library easier and even can greatly reduce development speed. Also are
we going to add new features to stable releases using this workflow with
single composition layer?

> Testing master while keeping stable. Given the ability to conditional
what source of openstack bits, which versions of manifests we can start
testing both master and keep health on stable. This would help accelerate
both fuel development and deploying and testing development versions of
openstack

I'm sorry, but I don't see how we can accelerate things by making
composition layer more and more complicated. If we're going to run CI and
swarm for all of the supported releases on the ISO, that would rather
decrease speed of development and testing drastically. Also aren't we
"testing both master and keep health on stable" right now by running tests
for master and stable versions of Fuel?

> Deploying stable and upgrading later. Again given the ability to deploy
multiple OpenStack versions within the same Fuel version, teams focused on
upgrades can take advantage of the latest enhancements in fuel to work the
upgrade process more easily, as an added benefit this would eventually lead
to better support for end user upgrades too.

Using the same composition layers is not required for this. Also how it
differs from using the current upgrade procedure? When you have, for
instance, 7.0 release and then upgrade to 8.0, so basically result is the
same - you have two releases in Fuel, 2 directories with manifests, 2 repos
with packages.

> Deploying older versions, in the odd case that we need to take advantage
of older OpenStack releases like in the case of Kilo with a newer version
of Fuel we can easily maintain that version too as we can keep the older
cases around in the composition layer with out adding much burden on the
other components.

Using the same composition layers is not required for this, "we can keep
the older cases around" in the composition layer of previous version.

Also, how many releases we're going to support? All of them starting from
Kilo? What about ISO size? What about CI, infra (required HW), acceptance
testing, etc impact?

Regards,
Alex



On Thu, Feb 11, 2016 at 6:57 AM, Andrew Woodward  wrote:

> Right now master (targeting 9.0) is still deploying liberty and there is
> active work going on to support both Kilo and Mitaka. On the review queue
> are changes that would make fuel-library in-compatible with the prior
> (liberty) openstack release. However I think if we extend a little bit of
> effort we can keep some semblance of "support" while creating a pattern for
> the Kilo support to continue to use. At the same time this pattern can help
> us test parallel versions as we move through openstack releases and should
> reduce occurrences of our CI freeze/merge parties
>
> What is this magic pattern? Well its already present, and all be it not
> designed for this I think we could quickly make it work. We use the release
> fixture already present in fuel. Originally designed to work for upgrades,
> we could reuse this within the same fuel release to control various aspects
> needed to deploy a separate openstack version.
>
> What we need to support multiple OpenStack versions:
> 1) Packge repo's that contain the relevant bits. CHECK, this can be
> toggled with a new release  [1][2]
> 2) can point to different Puppet modules CHECK, also in toggled release
>  [3]
> 3) Composition layer that supports calls to different puppet-openstack
> modules, WIP, it still needs work, but can be done [4]
>
> So what is open? The composition layer.
>
> Currently, we just abandon support for previous versions in the
> composition layer and leave them to only be monuments in the
> stable/ series branches for maintenance. If we instead started
> making changes (forwards or backwards that) change the calls based on the
> openstack version [5] then we would be able to change the calls based 

Re: [openstack-dev] [Fuel][library] relax downstream policy for puppet modules managed by librarian

[Aleksandr Didenko]

Hi,

one of cons: there might be delay in time when we need to apply a patch to
upstream project and thus fork some project (needs some time to prepare
patch to projects, get reviews and approves, etc). Having said that I vote
for "lazy downstreaming".

Regards,
Alex

On Fri, Jan 29, 2016 at 10:12 AM, Bogdan Dobrelya 
wrote:

> This is a continuation of the forked discussion [0].
>
> The idea is to relax Fuel-library downstream policy and implement a
> "lazy downstreaming", which is to not create a downstream fork of a
> puppet module referenced in the librarian Puppetfile unless we have to
> do so.
>
> The relaxed policy would unblock an upstream switching of the rabbitmq
> [1] and corosync [2] modules as well.
>
> Pros:
> - Reduced costs of maintaining downstream forks because of the laziness
> of the forking process. This much better distributes load to Fuel
> engineering and HW resources in time. Even though related tasks may be
> one-time, HW resources and network bandwidth shall not be wasted for
> keeping and cloning unnecessary forks (unless we have no choice but to
> fork things)
> -  A module might remain as direct upstream reference in the Puppetfile
> for quite a while. This as well shows an amount of the hidden technical
> debt in much more clean representation - downstream vs upstream refs in
> the Puppetfile.
> - Some generic, non OpenStack puppet modules will barely require
> backporting of separate patches by older tags as they work just
> straightforward and the latest version works for every supported Fuel
> release as well. Those would save resources because of the laziness of
> the relaxed process will never require to create downstream forks for
> such modules!
>
> Cons:
> - I see none. For "unlucky" modules, the *same* amount of work shall be
> done anyway, although with the lazy process in place it will be
> distributed in time much better.
>
> [0]
> http://lists.openstack.org/pipermail/openstack-dev/2016-January/085249.html
> [1] https://review.openstack.org/#/c/271217
> [2] https://review.openstack.org/#/c/273036/
>
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Overriding and removing VIPs from plugins

[Aleksandr Didenko]

> How are we handling this now for multiple plugins?

OK, so right now we can only add VIPs to array, we can't remove them. So
overriding would disable such possibility of adding VIPs from different
plugins. But with [0] patch it will be still possible to add and to remove
by providing empty array. But we'll have the problem with multiple plugins
in such case.
I've changed my mind :) I vote for leaving as in [0] since it's less
destructive comparing to what we have now.

Regards,
Alex

[0] https://review.openstack.org/#/c/273169/1

On Thu, Jan 28, 2016 at 5:23 PM, Aleksandr Didenko <adide...@mirantis.com>
wrote:

> Well, with merging instead of overriding, I believe, this problem with
> multiple plugins still exists, right? How are we handling this now for
> multiple plugins?
> Since VIPs is array I also vote for overriding, since merging it could be
> a pain (how do you remove existing element during array merge?).
>
> On Thu, Jan 28, 2016 at 5:00 PM, Aleksey Kasatkin <akasat...@mirantis.com>
> wrote:
>
>> Roman, please provide more details on how VIPs are overridden. And how do
>> you deal with multiple plugins.
>>
>>
>> Aleksey Kasatkin
>>
>>
>> On Thu, Jan 28, 2016 at 5:58 PM, Aleksey Kasatkin <akasat...@mirantis.com
>> > wrote:
>>
>>> Good idea, overally.
>>>
>>> How about multiple plugins? We don't have any sequence or priorities
>>> between them.
>>> What if one plugin assumes that VIP is present but other one wants to
>>> remove it?
>>>
>>>
>>> Aleksey Kasatkin
>>>
>>>
>>> On Thu, Jan 28, 2016 at 4:02 PM, Sergii Golovatiuk <
>>> sgolovat...@mirantis.com> wrote:
>>>
>>>> +1 to overriding VIPs
>>>>
>>>> --
>>>> Best regards,
>>>> Sergii Golovatiuk,
>>>> Skype #golserge
>>>> IRC #holser
>>>>
>>>> On Thu, Jan 28, 2016 at 12:04 PM, Vladimir Kuklin <vkuk...@mirantis.com
>>>> > wrote:
>>>>
>>>>> +1 to overriding VIPs
>>>>>
>>>>> On Thu, Jan 28, 2016 at 1:43 PM, Roman Prykhodchenko <m...@romcheg.me>
>>>>> wrote:
>>>>>
>>>>>> Folks,
>>>>>>
>>>>>> currently merge policy for network roles only allows to add new VIPs
>>>>>> to already existing roles [1]. If a plugin supplies a VIP with a name 
>>>>>> that
>>>>>> already exists but with different parameters, Nailgun will not allow to 
>>>>>> do
>>>>>> that. We faced a need to override configuration of several VIPs by
>>>>>> completely removing them from network roles by supplying something like
>>>>>> [2]. To enable that I’ve made a temporary workaround [3] to the merging
>>>>>> policy that only handles one cornerstone.
>>>>>>
>>>>>> I’ve talked to ikalnitsky and we realized that this functionality of
>>>>>> merging new VIPs from plugins to an existing network role is actually 
>>>>>> wrong
>>>>>> and should be replaced by complete overriding. Let’s discuss this
>>>>>> possibility here.
>>>>>>
>>>>>>
>>>>>> References:
>>>>>>
>>>>>> 1.
>>>>>> https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/policy/merge.py#L55
>>>>>> 2. http://xsnippet.org/361361/
>>>>>> 3. https://review.openstack.org/#/c/273169/1
>>>>>>
>>>>>>
>>>>>> - romcheg
>>>>>>
>>>>>>
>>>>>>
>>>>>> __
>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>> Unsubscribe:
>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Yours Faithfully,
>>>>> Vladimir Kuklin,
>>>>> Fuel Library Tech Lead,
>>>>> Mirantis, Inc.
>>>>> +7 (495) 640-49-04
>>>>> +7 (926) 702-39-68
>>>>> Skype kuklinvv
>>>>> 35bk3, Vorontsovskaya Str.
>>>>> Moscow, Russia,
>>>>> www.mirantis.com <http://www.mirantis.ru/>
>>>>> www.mirantis.ru
>>>>> vkuk...@mirantis.com
>>>>>
>>>>>
>>>>> __
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> __
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Overriding and removing VIPs from plugins

[Aleksandr Didenko]

Well, with merging instead of overriding, I believe, this problem with
multiple plugins still exists, right? How are we handling this now for
multiple plugins?
Since VIPs is array I also vote for overriding, since merging it could be a
pain (how do you remove existing element during array merge?).

On Thu, Jan 28, 2016 at 5:00 PM, Aleksey Kasatkin 
wrote:

> Roman, please provide more details on how VIPs are overridden. And how do
> you deal with multiple plugins.
>
>
> Aleksey Kasatkin
>
>
> On Thu, Jan 28, 2016 at 5:58 PM, Aleksey Kasatkin 
> wrote:
>
>> Good idea, overally.
>>
>> How about multiple plugins? We don't have any sequence or priorities
>> between them.
>> What if one plugin assumes that VIP is present but other one wants to
>> remove it?
>>
>>
>> Aleksey Kasatkin
>>
>>
>> On Thu, Jan 28, 2016 at 4:02 PM, Sergii Golovatiuk <
>> sgolovat...@mirantis.com> wrote:
>>
>>> +1 to overriding VIPs
>>>
>>> --
>>> Best regards,
>>> Sergii Golovatiuk,
>>> Skype #golserge
>>> IRC #holser
>>>
>>> On Thu, Jan 28, 2016 at 12:04 PM, Vladimir Kuklin 
>>> wrote:
>>>
 +1 to overriding VIPs

 On Thu, Jan 28, 2016 at 1:43 PM, Roman Prykhodchenko 
 wrote:

> Folks,
>
> currently merge policy for network roles only allows to add new VIPs
> to already existing roles [1]. If a plugin supplies a VIP with a name that
> already exists but with different parameters, Nailgun will not allow to do
> that. We faced a need to override configuration of several VIPs by
> completely removing them from network roles by supplying something like
> [2]. To enable that I’ve made a temporary workaround [3] to the merging
> policy that only handles one cornerstone.
>
> I’ve talked to ikalnitsky and we realized that this functionality of
> merging new VIPs from plugins to an existing network role is actually 
> wrong
> and should be replaced by complete overriding. Let’s discuss this
> possibility here.
>
>
> References:
>
> 1.
> https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/policy/merge.py#L55
> 2. http://xsnippet.org/361361/
> 3. https://review.openstack.org/#/c/273169/1
>
>
> - romcheg
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 35bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com 
 www.mirantis.ru
 vkuk...@mirantis.com


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] How to auto allocate VIPs for roles in different network node groups?

[Aleksandr Didenko]

Hi,

btw, we should also take into account the possibility to share networks in
Fuel-8.0. So if cluster is configured with shared public and management
networks then moving controllers into different network node groups (racks)
is fine and it will work out of the box [0] and we should not forbid such
configuration.

Regards,
Alex

[0] https://bugs.launchpad.net/fuel/+bug/1524320/comments/12

On Tue, Jan 19, 2016 at 9:42 AM, Aleksandr Didenko <adide...@mirantis.com>
wrote:

> Hi,
>
> I would also prefer second solution. The only real downside of it is the
> possibility to configure invalid cluster (for instance configure default
> "controller" roles in different racks). But such invalid configuration is
> possible only under some conditions:
> - User should configure multi-rack environment (network node groups). I'd
> say it's rather advanced Fuel usage and user most likely will follow our
> documentation, so we can describe possible problems in the documentation.
> - User should ignore notifications about possible problems from Fuel. I
> must say that this is quite possible when using CLI, because notifications
> should be checked manually in this case.
>
> Solution #1 is much safer, of course. But for me it looks like "let's
> forbid as much as we can just to avoid any risks". I prefer to give Fuel
> users a choice here which is possible only in second solution.
>
> > What if neither of node is in default group? Still use default group?
> > And prey that some third-party plugin will handle this case properly?
>
> No, let's put a warning for user. I don't think that forbidding is the
> proper way of handling such situations. Especially when we're not going to
> forbid such setup in 9.0.
>
> > Default is just pre-created nodegroup and that's it, so there's nothing
> special in it.
>
> Not quite. Default groups is the group where Fuel node is connected to.
>
> > We don't support load-balancing for nodes in different racks out-of-box.
>
> True. But we're going block deployment of roles that share VIP (created
> from plugin, for instance) even when no load-balancing is involved at all -
> just to be safe.
>
> Regards,
> Alex
>
> On Fri, Jan 15, 2016 at 10:50 AM, Bogdan Dobrelya <bdobre...@mirantis.com>
> wrote:
>
>> On 15.01.2016 10:19, Aleksandr Didenko wrote:
>> > Hi,
>> >
>> > We need to come up with some solution for a problem with VIP generation
>> > (auto allocation), see the original bug [0].
>> >
>> > The main problem here is: how do we know what exactly IPs to auto
>> > allocate for VIPs when needed roles are in different nodegroups (i.e. in
>> > different IP networks)?
>> > For example 'public_vip' for 'controller' roles.
>> >
>> > Currently we have two possible solutions.
>> >
>> > 1) Fail early in pre-deployment check (when user hit "Deploy changes")
>> > with error about inability to auto allocate VIP for nodes in different
>> > nodegroups (racks). So in order to run deploy user has to put all roles
>> > with the same VIPs in the same nodegroups (for example: all controllers
>> > in the same nodegroup).
>> >
>> > Pros:
>> >
>> >   * VIPs are always correct, they are from the same network as nodes
>> > that are going to use them, thus user simply can't configure invalid
>> > VIPs for cluster and break deployment
>> >
>> > Cons:
>> >
>> >   * hardcoded limitation that is impossible to bypass, does not allow to
>> > spread roles with VIPs across multiple racks even if it's properly
>> > handled by Fuel Plugin, i.e. made so by design
>>
>> That'd be no good at all.
>>
>> >
>> >
>> > 2) Allow to move roles that use VIPs into different nodegroups, auto
>> > allocate VIPs from "default" nodegroup and send an alert/notification to
>> > user that such configuration may not work and it's up to user how to
>> > proceed (either fix config or deploy at his/her own risk).
>>
>> It seems we have not much choice then, but use the option 2
>>
>> >
>> > Pros:
>> >
>> >   * relatively simple solution
>> >
>> >   * impossible to break VIP serialization because in the worst case we
>> > allocate VIPs from default nodegroup
>> >
>> > Cons:
>> >
>> >   * user can deploy invalid environment that will fail during deployment
>> > or will not operate properly (for example when public_vip is not
>> > able to migrate to controller from different

Re: [openstack-dev] [fuel][plugin] node_role only need when attribute false - where is the fuel plugin parser code?

[Aleksandr Didenko]

Whoops, forgot to add a link, sorry.. Here it is

[0] http://paste.openstack.org/show/484552/

On Thu, Jan 21, 2016 at 1:24 PM, Aleksandr Didenko <adide...@mirantis.com>
wrote:

> Hi,
>
> I'm working on a plugin for 8.0 atm and this is how it worked for me [0].
> It's a restriction not for node role, it's for other plugin setting, but I
> suppose it should work in your case as well.
> So in general it should look like:
>
> condition: "settings:plugin_name.attribute_name.value == false"
>
> Regards,
> Alex
>
> On Wed, Jan 20, 2016 at 8:07 PM, Matthew Mosesohn <mmoses...@mirantis.com>
> wrote:
>
>> Hi Nikolas,
>>
>> I'm not exactly sure about your case, but you should try something like
>> this:
>> https://github.com/openstack/fuel-plugin-detach-keystone/blob/master/node_roles.yaml#L14-L15
>> restrictions:
>> - condition: "settings:opendaylight_plugin:use_external_odl == false"
>> - message: "OpenDaylight role can only be used without external ODL"
>>
>> On Wed, Jan 20, 2016 at 9:41 PM, Nikolas Hermanns <
>> nikolas.herma...@ericsson.com> wrote:
>>
>>> Hey,
>>>
>>> I am developing on a fuel plugin at the moment.
>>> (fuel-plugin-opendaylight)
>>> In node_roles.yaml I would like to define something similar to:
>>> opendaylight:
>>>   limits:
>>> max: 1
>>> min: if "attributes:use_external_odl == true" then 0 else 1
>>>
>>> attributes:use_external_odl comes from the environment_config. Is such
>>> thing possible? And where is the code where that logic is actually build
>>> up. Which repo is having that?
>>>
>>> BR Nikolas
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel][plugin] node_role only need when attribute false - where is the fuel plugin parser code?

[Aleksandr Didenko]

Hi,

I'm working on a plugin for 8.0 atm and this is how it worked for me [0].
It's a restriction not for node role, it's for other plugin setting, but I
suppose it should work in your case as well.
So in general it should look like:

condition: "settings:plugin_name.attribute_name.value == false"

Regards,
Alex

On Wed, Jan 20, 2016 at 8:07 PM, Matthew Mosesohn 
wrote:

> Hi Nikolas,
>
> I'm not exactly sure about your case, but you should try something like
> this:
> https://github.com/openstack/fuel-plugin-detach-keystone/blob/master/node_roles.yaml#L14-L15
> restrictions:
> - condition: "settings:opendaylight_plugin:use_external_odl == false"
> - message: "OpenDaylight role can only be used without external ODL"
>
> On Wed, Jan 20, 2016 at 9:41 PM, Nikolas Hermanns <
> nikolas.herma...@ericsson.com> wrote:
>
>> Hey,
>>
>> I am developing on a fuel plugin at the moment. (fuel-plugin-opendaylight)
>> In node_roles.yaml I would like to define something similar to:
>> opendaylight:
>>   limits:
>> max: 1
>> min: if "attributes:use_external_odl == true" then 0 else 1
>>
>> attributes:use_external_odl comes from the environment_config. Is such
>> thing possible? And where is the code where that logic is actually build
>> up. Which repo is having that?
>>
>> BR Nikolas
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Relieving CI/gate jenkins bottleneck

[Aleksandr Didenko]

Hi,

> I also think 3.3 is the version that ships with 14.04.

3.4.3 is shipped with Ubuntu-14.04. I think 3.4, 3.8 and 4 should be enough.

Regards,
Alex

On Wed, Jan 20, 2016 at 6:38 PM, Sergii Golovatiuk  wrote:

> +1 for 3.3, 3.4, 3.8 and 4
>
>
> --
> Best regards,
> Sergii Golovatiuk,
> Skype #golserge
> IRC #holser
>
> On Wed, Jan 20, 2016 at 6:12 PM, Alex Schultz 
> wrote:
>
>> On Wed, Jan 20, 2016 at 9:02 AM, Matthew Mosesohn
>>  wrote:
>> > Hi all,
>> >
>> > Unit tests on CI and gate bottleneck are really slowing down commit
>> > progress. We recently had a meeting to discuss possible ways to improve
>> > this, including symlinks, caching git repositories, etc, but one thing
>> we
>> > can do much faster is to simply disable 3.3-3.7 puppet jobs. We don't
>> deploy
>> > Fuel 9.0 (or 8.0) on earlier Puppet versions, so what value is there to
>> the
>> > checks? I propose we remove these tests, and hopefully we will see some
>> > immediate relief.
>> >
>>
>> How about we reduce to 3.3, 3.4, 3.8 and 4?  We would remove  3.6 and
>> 3.7 which would reduce the number of jobs by a third  The goal of
>> keeping the others was to ensure that if/when we are able to install
>> fuel-library without our version of puppet that a user could use
>> whatever version their environment has. There were some changes
>> between 3.3 and 3.4 (if I remember correctly) so we should keep
>> checking that as it's also the oldest version supported by the
>> upstream puppet openstack modules.  I also think 3.3 is the version
>> that ships with 14.04.  Additionally we used 3.4 in fuel 7 and below
>> so we should keep those around.
>>
>> -Alex
>>
>> > Best Regards,
>> > Matthew Mosesohn
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] How to auto allocate VIPs for roles in different network node groups?

[Aleksandr Didenko]

Hi,

I would also prefer second solution. The only real downside of it is the
possibility to configure invalid cluster (for instance configure default
"controller" roles in different racks). But such invalid configuration is
possible only under some conditions:
- User should configure multi-rack environment (network node groups). I'd
say it's rather advanced Fuel usage and user most likely will follow our
documentation, so we can describe possible problems in the documentation.
- User should ignore notifications about possible problems from Fuel. I
must say that this is quite possible when using CLI, because notifications
should be checked manually in this case.

Solution #1 is much safer, of course. But for me it looks like "let's
forbid as much as we can just to avoid any risks". I prefer to give Fuel
users a choice here which is possible only in second solution.

> What if neither of node is in default group? Still use default group?
> And prey that some third-party plugin will handle this case properly?

No, let's put a warning for user. I don't think that forbidding is the
proper way of handling such situations. Especially when we're not going to
forbid such setup in 9.0.

> Default is just pre-created nodegroup and that's it, so there's nothing
special in it.

Not quite. Default groups is the group where Fuel node is connected to.

> We don't support load-balancing for nodes in different racks out-of-box.

True. But we're going block deployment of roles that share VIP (created
from plugin, for instance) even when no load-balancing is involved at all -
just to be safe.

Regards,
Alex

On Fri, Jan 15, 2016 at 10:50 AM, Bogdan Dobrelya <bdobre...@mirantis.com>
wrote:

> On 15.01.2016 10:19, Aleksandr Didenko wrote:
> > Hi,
> >
> > We need to come up with some solution for a problem with VIP generation
> > (auto allocation), see the original bug [0].
> >
> > The main problem here is: how do we know what exactly IPs to auto
> > allocate for VIPs when needed roles are in different nodegroups (i.e. in
> > different IP networks)?
> > For example 'public_vip' for 'controller' roles.
> >
> > Currently we have two possible solutions.
> >
> > 1) Fail early in pre-deployment check (when user hit "Deploy changes")
> > with error about inability to auto allocate VIP for nodes in different
> > nodegroups (racks). So in order to run deploy user has to put all roles
> > with the same VIPs in the same nodegroups (for example: all controllers
> > in the same nodegroup).
> >
> > Pros:
> >
> >   * VIPs are always correct, they are from the same network as nodes
> > that are going to use them, thus user simply can't configure invalid
> > VIPs for cluster and break deployment
> >
> > Cons:
> >
> >   * hardcoded limitation that is impossible to bypass, does not allow to
> > spread roles with VIPs across multiple racks even if it's properly
> > handled by Fuel Plugin, i.e. made so by design
>
> That'd be no good at all.
>
> >
> >
> > 2) Allow to move roles that use VIPs into different nodegroups, auto
> > allocate VIPs from "default" nodegroup and send an alert/notification to
> > user that such configuration may not work and it's up to user how to
> > proceed (either fix config or deploy at his/her own risk).
>
> It seems we have not much choice then, but use the option 2
>
> >
> > Pros:
> >
> >   * relatively simple solution
> >
> >   * impossible to break VIP serialization because in the worst case we
> > allocate VIPs from default nodegroup
> >
> > Cons:
> >
> >   * user can deploy invalid environment that will fail during deployment
> > or will not operate properly (for example when public_vip is not
> > able to migrate to controller from different rack)
> >
> >   * which nodegroup to choose to allocate VIPs? default nodegroup?
> > random pick? in case of random pick troubleshooting may become
> > problematic
>
> Random choices aren't good IMHO, let's use defaults.
>
> >
> >   * waste of IPs - IP address from the network range will be implicitly
> > allocated and marked as used, even it's not used by deployment
> > (plugin uses own ones)
> >
> >
> > *Please also note that this solution is needed for 8.0 only.*In 9.0 we
> > have new feature for manual VIPs allocation [1]. So in 9.0, if we can't
> > auto allocate VIPs for some cluster configuration, we can simply ask
> > user to manually set those problem VIPs or move roles to the same
> > network node group (rack).
> >
> > So, guys, please feel 

[openstack-dev] [Fuel] How to auto allocate VIPs for roles in different network node groups?

[Aleksandr Didenko]

Hi,

We need to come up with some solution for a problem with VIP generation
(auto allocation), see the original bug [0].

The main problem here is: how do we know what exactly IPs to auto allocate
for VIPs when needed roles are in different nodegroups (i.e. in different
IP networks)?
For example 'public_vip' for 'controller' roles.

Currently we have two possible solutions.

1) Fail early in pre-deployment check (when user hit "Deploy changes") with
error about inability to auto allocate VIP for nodes in different
nodegroups (racks). So in order to run deploy user has to put all roles
with the same VIPs in the same nodegroups (for example: all controllers in
the same nodegroup).

Pros:

   - VIPs are always correct, they are from the same network as nodes that
   are going to use them, thus user simply can't configure invalid VIPs for
   cluster and break deployment

Cons:

   - hardcoded limitation that is impossible to bypass, does not allow to
   spread roles with VIPs across multiple racks even if it's properly handled
   by Fuel Plugin, i.e. made so by design


2) Allow to move roles that use VIPs into different nodegroups, auto
allocate VIPs from "default" nodegroup and send an alert/notification to
user that such configuration may not work and it's up to user how to
proceed (either fix config or deploy at his/her own risk).

Pros:

   - relatively simple solution


   - impossible to break VIP serialization because in the worst case we
   allocate VIPs from default nodegroup

Cons:

   - user can deploy invalid environment that will fail during deployment
   or will not operate properly (for example when public_vip is not able to
   migrate to controller from different rack)


   - which nodegroup to choose to allocate VIPs? default nodegroup? random
   pick? in case of random pick troubleshooting may become problematic


   - waste of IPs - IP address from the network range will be implicitly
   allocated and marked as used, even it's not used by deployment (plugin uses
   own ones)


*Please also note that this solution is needed for 8.0 only.* In 9.0 we
have new feature for manual VIPs allocation [1]. So in 9.0, if we can't
auto allocate VIPs for some cluster configuration, we can simply ask user
to manually set those problem VIPs or move roles to the same network node
group (rack).

So, guys, please feel free to share your thoughts on this matter. Any input
is greatly appreciated.

Regards,
Alex

[0] https://bugs.launchpad.net/fuel/+bug/1524320
[1] https://blueprints.launchpad.net/fuel/+spec/allow-any-vip
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Diagnostic snapshot generation is broken due to lack of disk space

[Aleksandr Didenko]

Hi,

> b) Make the snapshot location share the diskspace of /var/log?

+1 for that. And +1 for using hard links to save space during snapshot
creation.

Regards,
Alex

On Tue, Jan 12, 2016 at 12:12 PM, Artem Panchenko 
wrote:

> Hi,
>
> doesn't matter how /var partition is big, diagnostic snapshot still could
> occupy all its space if there are lots of logs on slave nodes. Although, we
> can try to control disk space usage by snapshot in shotgun, but IMHO it's
> much safer to keep all related to logs staff away from critical system
> files, so I'm voting for 2nd (b) option. By the way it will allow us to use
> hard links for fast copying of files before creating tarball, because one
> partition for storing logs and snapshot will be used.
>
> Thanks!
>
>
> On 12.01.16 12:47, Maciej Kwiek wrote:
>
> Hi!
>
> I need some advice on how to tackle this issue. There is a bug [1]
> describing the problem with creating a diagnostic snapshot. The issue is
> that /var/log has 100GB available, while /var (where diagnostic snapshot is
> being generated - /var/www/nailgun/dump/fuel-snapshot according to [2]) has
> 10GB available, so dumping the logs can be an issue when logs size exceed
> free space in /var.
>
> There are several things we could do, but I am unsure on which course to
> take. Should we
> a) Allocate more disk space for /var/www (or for whole /var)?
> b) Make the snapshot location share the diskspace of /var/log?
> c) Something else? What?
>
> Please share your thoughts on this.
>
> Cheers,
> Maciej Kwiek
>
> [1] https://bugs.launchpad.net/fuel/+bug/1529182
> [2]
> https://github.com/openstack/fuel-web/blob/2855a9ba925c146b4802ab3cd2185f1dce2d8a6a/nailgun/nailgun/settings.yaml#L717
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> --
> Artem Panchenko
> QA Engineer
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Wipe of the nodes' disks

[Aleksandr Didenko]

> accurately wipe only partition table and do not touch any other data

As Andrew said already, in such case LVM meta data will remain on the hard
drive. So if you remove partition table, reboot the node (env reset), then
configure exactly the same partition table (like when you use the same
default disk allocation in Fuel), then Linux will find LVM info on the same
sectors of HDD on re-created partitions and re-assemble old LVM devices
automatically.

Regards,
Alex

On Tue, Dec 29, 2015 at 4:25 PM, Artur Svechnikov 
wrote:

> Thank you all for the answers.
> Well, it seems we should figure out how to do it another way. For now I
> see only one solution - accurately wipe only partition table and do not
> touch any other data. But this solution require huge changes in volume
> manager and fuel-agent, so I don't like this solution. I'm trying to find
> out better one so if you have any ideas please share it here.
>
> Best regards,
> Svechnikov Artur
>
> On Tue, Dec 29, 2015 at 4:34 PM, Sergii Golovatiuk <
> sgolovat...@mirantis.com> wrote:
>
>> Hi,
>>
>> Let me comment inline.
>>
>>
>> On Mon, Dec 28, 2015 at 7:06 PM, Andrew Woodward 
>> wrote:
>>
>>> In order to ensure that LVM can be configured as desired, its necessary
>>> to purge them and then reboot the node, otherwise the partitioning commands
>>> will most likely fail on the next attempt as they will be initialized
>>> before we can start partitioning the node. Hence, when a node is removed
>>> from the environment, it is supposed to have this data destroyed. Since
>>> it's a running system, the most effective way was to blast the first 1Mb of
>>> each partition. (with out many more reboots)
>>>
>>> As to the fallback to SSH, there are two times we use this process, with
>>> the node reboot (after cobbler/IBP finishes), and with the wipe as we are
>>> discussing here. These are for the odd occurrences of the nodes failing to
>>> restart after the MCO command. I don't think anyone has had much success
>>> trying to figure out why this occurs, but I've seen nodes get stuck in
>>> provisioning and remove in multiple environments using 6.1 where they
>>> managed to break the SSH Fallback. It would occur around 1/20 nodes
>>> seemingly randomly. So with the SSH fallback I nearly never see the failure
>>> in node reboot.
>>>
>>
>> If we talk about 6.1-7.0 release there shouldn't be any problems with mco
>> reboot. SSH fallback must be deprecated at all.
>>
>>
>>>
>>>
>>
>>>
>>
>>>
>>
>>> On Thu, Dec 24, 2015 at 6:28 AM Alex Schultz 
>>> wrote:
>>>
 On Thu, Dec 24, 2015 at 1:29 AM, Artur Svechnikov
  wrote:
 > Hi,
 > We have faced the issue that nodes' disks are wiped after stop
 deployment.
 > It occurs due to the logic of nodes removing (this is old logic and
 it's not
 > actual already as I understand). This logic contains step which calls
 > erase_node[0], also there is another method with wipe of disks [1].
 AFAIK it
 > was needed for smooth cobbler provision and ensure that nodes will
 not be
 > booted from disk when it shouldn't. Instead of cobbler we use IBP from
 > fuel-agent where current partition table is wiped before provision
 stage.
 > And use disks wiping for insurance that nodes will not booted from
 disk
 > doesn't seem good solution. I want to propose not to wipe disks and
 simply
 > unset bootable flag from node disks.

>>>
>> Disks must be wiped as boot flag doesn't guarantee anything. If bootlag
>> is not set, BIOS will ignore ignore the device in boot-order. More over, 2
>> partitions may have bootflag or operator may set to skip boot-order in BIOS.
>>
>> >
 > Please share your thoughts. Perhaps some other components use the
 fact that
 > disks are wiped after node removing or stop deployment. If it's so,
 then
 > please tell about it.
 >
 > [0]
 >
 https://github.com/openstack/fuel-astute/blob/master/lib/astute/nodes_remover.rb#L132-L137
 > [1]
 >
 https://github.com/openstack/fuel-astute/blob/master/lib/astute/ssh_actions/ssh_erase_nodes.rb
 >

 I thought the erase_node[0] mcollective action was the process that
 cleared a node's disks after their removal from an environment. When
 do we use the ssh_erase_nodes?  Is it a fall back mechanism if the
 mcollective fails?  My understanding on the history is based around
 needing to have the partitions and data wiped so that the LVM groups
 and other partition information does not interfere with the
 installation process the next time the node is provisioned.  That
 might have been a side effect of cobbler and we should test if it's
 still an issue for IBP.

>>>
>> Since we do not use classical provision anymore, we have mco connection
>> all the time. During IBP we have it as part of bootstrap, after reboot, mco
>> is 

Re: [openstack-dev] [Fuel] Wipe of the nodes' disks

[Aleksandr Didenko]

Hi,

> I want to propose not to wipe disks and simply unset bootable flag from
node disks.

AFAIK, removing bootable flag does not guarantee that system won't be
booted from the local drive. This is why erase_node is needed.

Regards,
Alex

On Fri, Dec 25, 2015 at 8:59 AM, Artur Svechnikov 
wrote:

> > When do we use the ssh_erase_nodes?
>
> It's used in stop_deployment provision stage [0] and for control reboot
> [1].
>
> > Is it a fall back mechanism if the mcollective fails?
>
> Yes it's like fall back mechanism, but it's used always [2].
>
> > That might have been a side effect of cobbler and we should test if it's
> > still an issue for IBP.
>
> As I can see from the code partition table always is wiped before
> provision [3].
>
> [0]
> https://github.com/openstack/fuel-astute/blob/master/lib/astute/provision.rb#L387-L396
> [1]
> https://github.com/openstack/fuel-astute/blob/master/lib/astute/provision.rb#L417-L425
> [2]
> https://github.com/openstack/fuel-astute/blob/master/lib/astute/provision.rb#L202-L208
> [3]
> https://github.com/openstack/fuel-agent/blob/master/fuel_agent/manager.py#L194-L197
>
> Best regards,
> Svechnikov Artur
>
> On Thu, Dec 24, 2015 at 5:27 PM, Alex Schultz 
> wrote:
>
>> On Thu, Dec 24, 2015 at 1:29 AM, Artur Svechnikov
>>  wrote:
>> > Hi,
>> > We have faced the issue that nodes' disks are wiped after stop
>> deployment.
>> > It occurs due to the logic of nodes removing (this is old logic and
>> it's not
>> > actual already as I understand). This logic contains step which calls
>> > erase_node[0], also there is another method with wipe of disks [1].
>> AFAIK it
>> > was needed for smooth cobbler provision and ensure that nodes will not
>> be
>> > booted from disk when it shouldn't. Instead of cobbler we use IBP from
>> > fuel-agent where current partition table is wiped before provision
>> stage.
>> > And use disks wiping for insurance that nodes will not booted from disk
>> > doesn't seem good solution. I want to propose not to wipe disks and
>> simply
>> > unset bootable flag from node disks.
>> >
>> > Please share your thoughts. Perhaps some other components use the fact
>> that
>> > disks are wiped after node removing or stop deployment. If it's so, then
>> > please tell about it.
>> >
>> > [0]
>> >
>> https://github.com/openstack/fuel-astute/blob/master/lib/astute/nodes_remover.rb#L132-L137
>> > [1]
>> >
>> https://github.com/openstack/fuel-astute/blob/master/lib/astute/ssh_actions/ssh_erase_nodes.rb
>> >
>>
>> I thought the erase_node[0] mcollective action was the process that
>> cleared a node's disks after their removal from an environment. When
>> do we use the ssh_erase_nodes?  Is it a fall back mechanism if the
>> mcollective fails?  My understanding on the history is based around
>> needing to have the partitions and data wiped so that the LVM groups
>> and other partition information does not interfere with the
>> installation process the next time the node is provisioned.  That
>> might have been a side effect of cobbler and we should test if it's
>> still an issue for IBP.
>>
>>
>> Thanks,
>> -Alex
>>
>> [0]
>> https://github.com/openstack/fuel-astute/blob/master/mcagents/erase_node.rb
>>
>> > Best regards,
>> > Svechnikov Artur
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Nominate Artem Roma for fuel-ostf core

[Aleksandr Didenko]

+1

On Wed, Dec 23, 2015 at 4:21 PM, Andrey Sledzinskiy <
asledzins...@mirantis.com> wrote:

> +1
>
> On Wed, Dec 23, 2015 at 5:05 PM, Tatyana Leontovich <
> tleontov...@mirantis.com> wrote:
>
>> Hi guys,
>> I'd like to nominate Artem Roma [0] for fuel-ostf core.
>>
>> Artem  cares about fuel-ostf more then 2 years, always provide a great
>> reviews(always thorough and consistent and comes in time), extend unit
>> tests coverages and provide a lot of bug fixes and improvements there.
>>
>> Please, vote for Artem!
>>
>>
>> http://stackalytics.com/?user_id=aroma-x=all_type=all=fuel-ostf
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Thanks,
> Andrey Sledzinskiy
> QA Engineer,
> Mirantis, Kharkiv
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Nominate Artem Panchenko for fuel-qa core

[Aleksandr Didenko]

+1

On Wed, Dec 23, 2015 at 2:52 PM, Tatyana Leontovich <
tleontov...@mirantis.com> wrote:

> Absolutely agree
>
> +1
>
>
>
>
> On Wed, Dec 23, 2015 at 3:42 PM, Andrey Sledzinskiy <
> asledzins...@mirantis.com> wrote:
>
>> Hi guys,
>> I'd like to nominate Artem Panchenko [0] for fuel-qa core.
>>
>> Artem has great technical expertise in fuel-qa and he developed a lot of
>> vital parts of framework. His first place in a number of commits is a good
>> proof of that.
>> His reviews are always thorough and consistent.
>> Please, vote for Artem!
>>
>> [0]
>> http://stackalytics.com/?user_id=apanchenko-8=all_type=all=fuel-qa
>>
>> --
>> Thanks,
>> Andrey Sledzinskiy
>> QA Engineer,
>> Mirantis, Kharkiv
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Multirack] Are we going to support Ceph Multirack deployments?

[Aleksandr Didenko]

Hi,

just finished deployment of multirack env with ceph and external LB (not a
standard multi-rack deployment scheme, actually, but still):

http://paste.openstack.org/show/482610/

As you can see it works just fine in multirack.

Also, we're running 'deploy_ceph_ha_nodegroups' system test as a part of
'thread_7' group in our swarm tests, so ceph in multirack is covered with
automated tests as well.

Regards,
Alex

On Wed, Dec 23, 2015 at 5:29 PM, Roman Sokolkov 
wrote:

> + Konstantin and Alex
>
> Can someone give a quick highlight on that question?
>
> On Tue, Dec 22, 2015 at 8:17 PM, Roman Sokolkov 
> wrote:
>
>> Folks,
>>
>> multirack is going to be a part of MOS 8.0.
>>
>> But i can not find any info about Ceph?
>>
>> Technically Ceph supports Multiple L3 networks [1].
>>
>> [1] -
>> http://docs.ceph.com/docs/hammer/rados/configuration/network-config-ref/#network-config-settings
>>
>> --
>> Roman Sokolkov,
>> Deployment Engineer,
>> Mirantis, Inc.
>> Skype rsokolkov,
>> rsokol...@mirantis.com
>>
>
>
>
> --
> Roman Sokolkov,
> Deployment Engineer,
> Mirantis, Inc.
> Skype rsokolkov,
> rsokol...@mirantis.com
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][library] CI gate for regressions detection in deployment data

[Aleksandr Didenko]

Hi,

btw, right now we have 33 outdated astute.yaml fixtures for noop rspec
tests [0] - and this number is based on a single parameter of
network_metadata['vips'] hash, actual amount of outdated fixtures could be
bigger. So I've registered a new BP [1] to create a script that will
generate up-to-date fixtures on demand and/or periodically.

Regards,
Alex

[0] http://paste.openstack.org/show/482508/
[1]
https://blueprints.launchpad.net/fuel/+spec/deployment-dryrun-fixtures-generator


On Mon, Dec 7, 2015 at 11:51 AM, Bogdan Dobrelya <bdobre...@mirantis.com>
wrote:

> On 02.12.2015 17:03, Bogdan Dobrelya wrote:
> > On 01.12.2015 11:28, Aleksandr Didenko wrote:
> >> Hi,
> >>
> >>> pregenerated catalogs for the Noop tests to become the very first
> >>> committed state in the data regression process has to be put in the
> >>> *separate repo*
> >>
> >> +1 to that, we can put this new repo into .fixtures.yml
> >>
> >>> note, we could as well move the tests/noop/astute.yaml/ there
> >>
> >> +1 here too, astute.yaml files are basically configuration fixtures, we
> >> can put them into .fixtures.yml as well
>
> Folks, the patch to create the fuel-noop-fixtures [0] is in trouble.
> I'm not sure I've answered Andreas's questions correct:
>
> - Would it be OK to keep Noop tests fixtures for fuel-library as a
> separate Fuel-related repo but *not* as a part of the Fuel project?
>
> - Should we require the contribution license agreement for fixtures
> which are only be used by tests?
>
> [0] https://review.openstack.org/252992
>
> >
> > I found the better -and easier for patch authors- way to use the data
> > regression checks. Originally suggested workflow was:
> >
> > 1.
> > "The check should be done for every modular component (aka deployment
> > task). Data generated in the noop catalog run for all classes and
> > defines of a given deployment task should be verified against its
> > "acknowledged" (committed) state."
> >
> > This part remains the same with the only comment that the astute.yaml
> > fixtures of deployment cases should be fetched from the
> > fuel-noop-fixtures repo. And the committed state for generated catalogs
> > should be
> > stored there as well.
> >
> > 2.
> > "And fail the test gate, if changes has been found, like new parameter
> > with a defined value, removed a parameter, changed a parameter's value."
> >
> > This should be changed as following:
> > - the data checks gate should be just a non voting helper for reviewers
> > and patch authors. The only its task would be to show inducted data
> > changes in a pretty and fast view to help accept/update/reject a patch
> > on review.
> > - the data checks gate job should fetch the committed data state from
> > the fuel-noop-fixtures repo and run regressions check with the patch
> > under review checked out on fuel-library repo.
> > - the Noop tests gate should be changed to fetch the astute.yaml
> > fixtures from the fuel-noop-fixtures repo in order to run noop tests as
> > usual.
> >
> > 3.
> > "In order to remove a regression, a patch author will have to add (and
> > reviewers should acknowledge) detected changes in the committed state of
> > the deployment data. This may be done manually, with a tool like [3] or
> > by a pre-commit hook, or even at the CI side!"
> >
> > Instead, the patch authors should do nothing additionally. Once accepted
> > with wf+1, the patch on reivew should be merged with a pre-commit zuul
> > hook (is it possible?). The hook should just regenerate catalogs with
> > the changes introduced by the patch and update the committed state of
> > data in the fuel-noop-fixtures repo. After that, the patch may be safely
> > merged to the fuel-library and everything will be up to date with the
> > committed data state.
> >
> > 4.
> > "The regression check should show the diff between committed state and a
> > new state proposed in a patch. Changed state should be *reviewed* and
> > accepted with a patch, to became a committed one. So the deployment data
> > will evolve with *only* approved changes. And those changes would be
> > very easy to be discovered for each patch under review process!"
> >
> > So this part would work even better now, with no additional actions
> > required from the review process sides.
> >
> >>
> >> Regards,
> >> Alex
> >>
> >>
> >> On Mon, Nov 30, 2015 at 1:03 PM, Bogdan Dobrelya <
> bdobre...@mir

Re: [openstack-dev] [Fuel] Proposal to Delay Docker Removal From Fuel Master Node

[Aleksandr Didenko]

Hi,

having or not having docker on Fuel master affects mostly Fuel node
deployment related scripts/manifests. If we need to fix a bug in Nailgun or
OSTF - we can fix it in their codebase and it does not really matter
whether we run Nailgin or OSTF inside docker or not. The same goes for all
the OpenStack deployment related manifests - fix will be the same in 9.0
without docker and in 8.0 with docker.
Removing docker may introduce a bunch of new bugs, but they will be only
9.0 specific and won't affect 8.0. So I don't think that removing docker in
master branch will affect bugfixing velocity for stable branch a lot.

Regards,
Alex

On Fri, Dec 18, 2015 at 1:45 AM, Sergii Golovatiuk  wrote:

> +1
>
> --
> Best regards,
> Sergii Golovatiuk,
> Skype #golserge
> IRC #holser
>
> On Thu, Dec 17, 2015 at 3:43 PM, Mike Scherbakov  > wrote:
>
>> If there are no concrete points why we should wait, I'm +1 to go ahead
>> with merges.
>>
>> On Thu, Dec 17, 2015 at 1:32 AM Oleg Gelbukh 
>> wrote:
>>
>>> Evgeniy,
>>>
>>> True, and I fully support merging this particular change as soon as
>>> possible, i.e. the moment the 'master' is open for 9.0 development.
>>>
>>> -Oleg
>>>
>>> On Thu, Dec 17, 2015 at 12:28 PM, Evgeniy L  wrote:
>>>
 Hi Oleg,

 With the same degree of confidence we can say that anything we have in
 the beginning of
 the release cycle is not urgent enough. We pushed early branching
 specifically for
 such big changes as Docker removal/Changing repos structures and
 merging invasive patches
 for new release features.

 Vladimir Kuklin,

 I'm not sure what do you mean by "fixing 2 different environments"?
 With environment without
 containers it will simplify debugging process.

 Thanks,

 On Wed, Dec 16, 2015 at 10:12 PM, Oleg Gelbukh 
 wrote:

> Hi
>
> Although I agree that it should be done, the removal of Docker doesn't
> seem an urgent feature to me. It is not blocking anything besides moving 
> to
> full package-based deployment of Fuel, as far as I understand. So it could
> be easily delayed for one milestone, especially if it is already almost
> done and submitted for review, so it could be merged fast before any other
> significant changes land in 'master' after it is open.
>
> --
> Best regards,
> Oleg Gelbukh
>
> On Wed, Dec 16, 2015 at 8:56 PM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com> wrote:
>
>> Vladimir,
>>
>> I have other activities planned for the time immediately after SCF
>> (separating UI from fuel-web, maybe it is even more invasive :-)) and it 
>> is
>> not a big deal to postpone this feature or another. I am against the
>> approach itself of postponing something because it is too invasive. If we
>> create stable branch master becomes open. That was our primary intention 
>> to
>> open master earlier than later when we decided to move stable branch
>> creation.
>>
>>
>>
>>
>> Vladimir Kozhukalov
>>
>> On Wed, Dec 16, 2015 at 8:28 PM, Vladimir Kuklin <
>> vkuk...@mirantis.com> wrote:
>>
>>> Vladimir
>>>
>>> I am pretty much for removing docker, but I do not think that we
>>> should startle our developers/QA folks with additional efforts on 
>>> fixing 2
>>> different environments. Let's just think from the point of development
>>> velocity here and at delay such changes for at least after NY. Because 
>>> if
>>> we do it immediately after SCF there will be a whole bunch of holidays 
>>> and
>>> Russian holidays are Jan 1st-10th and you (who is the SME for docker
>>> removal) will be offline. Do you really want to fix things instead of
>>> enjoying holidays?
>>>
>>> On Wed, Dec 16, 2015 at 4:09 PM, Evgeniy L  wrote:
>>>
 +1 to Vladimir Kozhukalov,

 Entire point of moving branches creation to SCF was to perform such
 changes as
 early as possible in the release, I see no reasons to wait for HCF.

 Thanks,

 On Wed, Dec 16, 2015 at 10:19 AM, Vladimir Kozhukalov <
 vkozhuka...@mirantis.com> wrote:

> -1
>
> We already discussed this and we have made a decision to move
> stable branch creation from HCF to SCF. There were reasons for this. 
> We
> agreed that once stable branch is created, master becomes open for new
> features. Let's avoid discussing this again.
>
> Vladimir Kozhukalov
>
> On Wed, Dec 16, 2015 at 9:55 AM, Bulat Gaifullin <
> bgaiful...@mirantis.com> wrote:
>
>> +1
>>
>> Regards,
>> Bulat Gaifullin
>> Mirantis 

Re: [openstack-dev] [Fuel][UX] Throw KVM\QEMU and leave Libvirt on Wizard

[Aleksandr Didenko]

Hi,

looks good to me.

Regards,
Alex

On Fri, Dec 18, 2015 at 10:17 AM, Andriy Popovych 
wrote:

> Hi fuelers,
>
> We want to throw KVM/QEMU options from Wizard and instead of them leave
> only one: Libvirt [0]. Libvirt option enables QEMU by default and there are
> still be possibility to change it on KVM in settings. It looks more
> logically because both QEMU\KVM are options for libvirt which manage them.
>
> What are you think about it?
>
> [0] https://review.openstack.org/#/c/258690
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] PostgreSQL 9.3 and JSON operations

[Aleksandr Didenko]

Hi,

> Downgrading for no reason could bring us to big trouble and bad user
experience

+1 to this. Let's keep PostgreSQL 9.3.

Regards,
Alex

On Mon, Dec 14, 2015 at 2:04 PM, Artem Silenkov 
wrote:

> Hello!
>
> Vote for update.
>
> 1. We have already shipped 9.3 in fuel-7.0. Downgrading such complicated
> package without any reason is not good thing at all. User experience could
> suffer a lot.
> 2. The next reason is tests. We have tested only 9.3, 9.2 was not tested
> at all. I'm sure we could bring serious regressions by downgrading,
> 3. Postgres-9.3 is not custom. It was taken from KOJI packages and
> backported without any modification. It means that this package is
> officially tested and supported by Fedora, which is good.
> 4. One shipped package more is not a huge burden for us. It was officially
> backported from official sources, tested and suits our need perfectly. Why
> do we need to play such dangerous games downgrading for no reasons?
>
> Let me notice that all packages are maintained by mos-packaging team now
> And we are perfectly ok with postgres-9.3.
>
> Downgrading for no reason could bring us to big trouble and bad user
> experience.
>
> Regards,
> Artem Silenkov
> ---
> MOs-Packaging
>
> On Mon, Dec 14, 2015 at 3:41 PM, Bartłomiej Piotrowski <
> bpiotrow...@mirantis.com> wrote:
>
>> On 2015-12-14 13:12, Igor Kalnitsky wrote:
>> > My opinion here is that I don't like that we're going to build and
>> > maintain one more custom package (just take a look at this patch [4]
>> > if you don't believe me), but I'd like to hear more opinion here.
>> >
>> > Thanks,
>> > Igor
>> >
>> > [1] https://bugs.launchpad.net/fuel/+bug/1523544
>> > [2] https://review.openstack.org/#/c/249656/
>> > [3] http://goo.gl/forms/Hk1xolKVP0
>> > [4] https://review.fuel-infra.org/#/c/14623/
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> I also think we should stay with what CentOS provides. Increasing
>> maintenance burden for something that can be implemented without bells
>> and whistles sounds like a no-go.
>>
>> Bartłomiej
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel-library] making task "apache" idempotent

[Aleksandr Didenko]

Hi,

I suppose disabling purge_configs is the only solution here. Running all
the configuration in a single task does not fit well into deployment
granularization philosophy.

Regards,
Alex

On Fri, Dec 11, 2015 at 12:13 PM, Dmitry Bilunov 
wrote:

> Hello.
>
> I am trying to pick a way of making puppet
> manifest fuel-library/deployment/puppet/osnailyfacter/modular/apache/apache.pp
> idempotent.
> During a normal execution flow, Fuel (fuel-astute, actually) would run
> "apache" task before running other tasks, which create virtual host
> configuration files inside /etc/apache2/sites-enabled/.
> "apache" task currently purges all configuration files, not created by it
> - see https://forge.puppetlabs.com/puppetlabs/apache#purge_configs; it
> seems reasonable to have this option enabled.
>
> However, fuel-astute runs all tasks in a separate "puppet apply"
> processes, which do not know anything about the state of each other. So I
> cannot use the intended (by puppetlabs-apache) way of regenerating the
> configs after the purge.
>
> One way to achieve it is moving the code, generating the configs, into
> apache.pp, so it will be re-run after each purge. However, it is
> complicated by the need of existence of various resources: services' files,
> users and packages. We cannot simply declare any of *::wsgi::apache classes
> inside apache.pp due to these dependencies.
>
> Another way is simply disabling purge_configs and purging manually
> everything that we don't need (like distro's apache mpm config).
>
> Which of these ways looks better? Are there any alternatives that I am
> missing?
> Thanks in advance.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Disable stringify_facts in 9.0

[Aleksandr Didenko]

Hi,

I agree, let's do this.

Regards,
Alex

On Fri, Dec 11, 2015 at 1:08 PM, Bartłomiej Piotrowski <
bpiotrow...@mirantis.com> wrote:

> Hi folks,
>
> my sense of aesthetics was slightly disturbed when I saw that the mounts
> fact[1] is implemented by joining mount points using a comma.
>
> It turns out that what Alex did is completely right as Puppet up to 3.8
> release has enabled stringify_facts by default. TLDR of that setting is
> that anything that is returned by Facter, ends up as a string:
>
> Facter says: ["/", "/boot", "/var", "/var/log"]
> (Cool, an array.)
>
> Puppet sees: "[\"/\", \"/boot\", \"/var\", \"/var/log\"]"
> (Not so cool.)
>
> In the result, any function that takes array as an argument either goes
> postal or aborts.
>
> I'd love to set stringify_facts to 'false' for 9.0 (as we're too late
> for that in 8.0). It is already done in Puppet 4 (which transition to
> I'd discuss maybe not this time) and apart from that, should decrease
> the number of WTF per minute for deployment engineers.
>
> What do you think?
>
> Bartłomiej
>
> [1] http://preview.tinyurl.com/q6ml86c
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] UI experience

[Aleksandr Didenko]

Hi,

that's great to know about this new button behaviour, because I was going
to file a bug few days ago when I was not able to find "Stop" button on UI.
But then I updated to a more recent ISO and the button "appeared" again.
Now I know what that was, thank you. And I can't tell that my own
experience was good. Hiding buttons is much more confusing than disabling
them, IMHO.

Regards,
Alex

On Fri, Dec 4, 2015 at 1:35 PM, Alexandra Morozova  wrote:

> Hi Sergii,
>
> Thank you very much for your feedback! We really appreciate it.
>
> I guess, you mean Reset
>  deployment and Stop
> deployment buttons?
>
> The idea here is that all potentially destructive action should be not
> exposed to the users, so it will prevent accidental clicking these buttons
> which will in most cases lead to broken OpenStack environment configuration.
>
> Reset button  is hidden when there were not any
> previous deploy operation with the OpenStack environment.
> Stop button  is hidden when it's impossible to stop
> deployment.
>
> Maybe, you are right, that showing disabled button will be better for the
> user, but let's keep in mind that it's better to hide some buttons, than to
> show a number of disabled ones. That's why the current behavior looks good
> for me.
>
> P.S. and again thanks for the feedback, we really need it.
>
> Best regards,
> Morozova Alexandra
> Software Developer, Mirantis, Inc.
> Skype: anarchistkasan
> #MorAle
> +48 514501223
> *Mirantis Poland Sp. z o.o.*
> *Szyperska St., 14/E6*
> *Poznań 61-754, Poland*
> www.mirantis.com
>
>
> On Fri, Dec 4, 2015 at 1:23 PM, Sergii Golovatiuk <
> sgolovat...@mirantis.com> wrote:
>
>> Hi,
>>
>> Recently I realised that some buttons disappear from UI in some
>> circumstances. However, these conditions are not clear to me. I suppose we
>> should change behaviour to block the button with proper caption message why
>> it's blocked instead of removing it from UI completely. It will improve UX
>> for advanced guys. What do you think?
>>
>> --
>> Best regards,
>> Sergii Golovatiuk,
>> Skype #golserge
>> IRC #holser
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Nominating Roman Prykhodchenko to python-fuelclient cores

[Aleksandr Didenko]

+1

On Wed, Dec 2, 2015 at 9:13 AM, Julia Aranovich 
wrote:

> +1
>
> On Tue, Dec 1, 2015 at 10:29 PM Sergii Golovatiuk <
> sgolovat...@mirantis.com> wrote:
>
>> +1
>>
>> --
>> Best regards,
>> Sergii Golovatiuk,
>> Skype #golserge
>> IRC #holser
>>
>> On Tue, Dec 1, 2015 at 6:15 PM, Aleksey Kasatkin 
>> wrote:
>>
>>> +1.
>>> No doubts. )
>>>
>>>
>>> Aleksey Kasatkin
>>>
>>>
>>> On Tue, Dec 1, 2015 at 5:49 PM, Dmitry Pyzhov 
>>> wrote:
>>>
 Guys,

 I propose to promote Roman Prykhodchenko to python-fuelclient cores. He
 is the main contributor and maintainer of this repo. And he did a great job
 making changes toward OpenStack recommendations. Cores, please reply with
 your +1/-1.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][library] CI gate for regressions detection in deployment data

[Aleksandr Didenko]

Hi,

> pregenerated catalogs for the Noop tests to become the very first
> committed state in the data regression process has to be put in the
> *separate repo*

+1 to that, we can put this new repo into .fixtures.yml

> note, we could as well move the tests/noop/astute.yaml/ there

+1 here too, astute.yaml files are basically configuration fixtures, we can
put them into .fixtures.yml as well

Regards,
Alex


On Mon, Nov 30, 2015 at 1:03 PM, Bogdan Dobrelya 
wrote:

> On 20.11.2015 17:41, Bogdan Dobrelya wrote:
> >> Hi,
> >>
> >> let me try to rephrase this a bit and Bogdan will correct me if I'm
> wrong
> >> or missing something.
> >>
> >> We have a set of top-scope manifests (called Fuel puppet tasks) that we
> use
> >> for OpenStack deployment. We execute those tasks with "puppet apply".
> Each
> >> task supposed to bring target system into some desired state, so puppet
> >> compiles a catalog and applies it. So basically, puppet catalog =
> desired
> >> system state.
> >>
> >> So we can compile* catalogs for all top-scope manifests in master branch
> >> and store those compiled* catalogs in fuel-library repo. Then for each
> >> proposed patch CI will compare new catalogs with stored ones and print
> out
> >> the difference if any. This will pretty much show what is going to be
> >> changed in system configuration by proposed patch.
> >>
> >> We were discussing such checks before several times, iirc, but we did
> not
> >> have right tools to implement such thing before. Well, now we do :) I
> think
> >> it could be quite useful even in non-voting mode.
> >>
> >> * By saying compiled catalogs I don't mean actual/real puppet catalogs,
> I
> >> mean sorted lists of all classes/resources with all parameters that we
> find
> >> during puppet-rspec tests in our noop test framework, something like
> >> standard puppet-rspec coverage. See example [0] for networks.pp task
> [1].
> >>
> >> Regards,
> >> Alex
> >>
> >> [0] http://paste.openstack.org/show/477839/
> >> [1]
> >>
> https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/openstack-network/networks.pp
> >
> > Thank you, Alex.
> > Yes, the composition layer is a top-scope manifests, known as a Fuel
> > library modular tasks [0].
> >
> > The "deployment data checks", is nothing more than comparing the
> > committed vs changed states of fixtures [1] of puppet catalogs for known
> > deployment paths under test with rspecs written for each modular task
> [2].
> >
> > And the *current status* is:
> > - the script for data layer checks now implemented [3]
> > - how-to is being documented here [4]
> > - a fix to make catalogs compilation idempotent submitted [5]
>
> The status update:
> - the issue [0] is the data regression checks blocker and is only the
> Noop tests specific. It has been reworked to not use custom facts [1].
> New uuid will be still generated each time in the catalog, but the
> augeas ensures it will be processed in idempotent way. Let's make this
> change [2] to the upstream puppet-nova as well please.
>
> [0] https://bugs.launchpad.net/fuel/+bug/1517915
> [1] https://review.openstack.org/251314
> [2] https://review.openstack.org/131710
>
> - pregenerated catalogs for the Noop tests to become the very first
> committed state in the data regression process has to be put in the
> *separate repo*. Otherwise, the stackalytics would go mad as that would
> be a 600k-liner patch to an OpenStack project, which is the Fuel-library
> now :)
>
> So, I'm planning to use the separate repo for the templates. Note, we
> could as well move the tests/noop/astute.yaml/ there. Thoughts?
>
> > - and there is my WIP branch [6] with the initial committed state of
> > deploy data pre-generated. So, you can checkout, make any test changes
> > to manifests and run the data check (see the README [4]). It works for
> > me, there is no issues with idempotent re-checks of a clean committed
> > state or tests failing when unexpected.
> >
> > So the plan is to implement this noop tests extention as a non-voting CI
> > gate after I make an example workflow update for developers to the
> > Fuel wiki. Thoughts?
> >
> > [0]
> >
> https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular
> > [1]
> >
> https://github.com/openstack/fuel-library/tree/master/tests/noop/astute.yaml
> > [2]
> https://github.com/openstack/fuel-library/tree/master/tests/noop/spec
> > [3] https://review.openstack.org/240015
> > [4]
> >
> https://github.com/openstack/fuel-library/blob/master/tests/noop/README.rst
> > [5] https://review.openstack.org/247989
> > [6] https://github.com/bogdando/fuel-library-1/commits/data_checks
> >
> >
>
>
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 

Re: [openstack-dev] [Fuel] CentOS7 Merging Plan

[Aleksandr Didenko]

Hi,

in your plan you need to have an item about updating ISO on fuel-library CI
gates.

Regards,
Alex

On Tue, Dec 1, 2015 at 4:11 PM, Sergii Golovatiuk 
wrote:

> Hi,
>
> On Tue, Dec 1, 2015 at 3:58 PM, Dmitry Teselkin 
> wrote:
>
>> Hello,
>>
>> We're almost got green BVT on custom CentOS7 ISO and it seems that it's
>> the time to discuss the plan how this feature could be merged.
>>
>> This is not the only one feature that is in a queue. Unfortunately,
>> almost any other feature will be broken if merged after CentOS7, so it
>> was decided to merge our changes last.
>>
>> This is not an official announcement, rather a notification letter to
>> start a discussion and find any objections.
>>
>> So the plan is:
>>
>> * merge all features that are going to be merged before Thusday, Dec 3
>>
>
> As far as I know some features won't be ready. They will require FFE or
> should be moved to 9.0
>
>
>> * call for merge freeze starting at Dec 3, due Dec 7
>> * rebase all CentOS7-related pathsets and resolve any conflicts with
>>   merged code (Dec 3)
>> * build custom ISO, pass BVT (and other tests) (Dec 3)
>> * merge all CentOS7-related patchsets at once (Dec 4)
>>
>
> I guess CI environment should be rebuilt
>
> * build an ISO and pass BVT again (Dec 4)
>> * run additional test during weekend (Dec 5, 6) to be sure that ISO
>>   good enough
>
>
>> According to this plan on Monday, Dec 7 we should either get CentOS7
>> based ISO, or revert all incompatible changes.
>>
>> --
>> Thanks,
>> Dmitry Teselkin
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel][puppet] Detached roles and globals.pp

[Aleksandr Didenko]

Hi,

I think something like this would be more flexible:

$swift_proxy_roles   = hiera('swift_proxy_roles', ['primary-controller',
'controller'])
$swift_storage_roles = hiera('swift_storage_roles', ['primary-controller',
'controller'])
# ...
$swift_nodes = get_nodes_hash_by_roles($network_metadata,
$swift_storage_roles)
$swift_proxies   = get_nodes_hash_by_roles($network_metadata,
$swift_proxy_roles)
$swift_proxy_caches  = get_nodes_hash_by_roles($network_metadata,
$swift_proxy_roles)

Regards,
Alex


On Wed, Nov 11, 2015 at 12:41 PM, Daniel Depaoli <
daniel.depa...@create-net.org> wrote:

> Hi all.
> I'm starting to resolve the todo at these line[1]. To solve this I think
> to hardcoded the role in the file, for example:
>
> *$swift_proxies = get_nodes_hash_by_roles($network_metadata,
> ['primary-swift-proxy', 'swift-proxy']) ? {*
> *true => get_nodes_hash_by_roles($network_metadata,
> ['primary-swift-proxy', 'swift-proxy']),*
> *false => get_nodes_hash_by_roles($network_metadata,
> ['primary-controller', 'controller']),*
> *}*
>
>
> Is this the right way or do you suggest something more clean?
>
> Thanks
>
> [1]
> https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/globals/globals.pp#L236:L242
>
> --
> 
> Daniel Depaoli
> CREATE-NET Research Center
> Smart Infrastructures Area
> Junior Research Engineer
> 
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Fuel-QA][Fuel-TechDebt] Code Quality: Do Not Hardcode - Fix Things Instead

[Aleksandr Didenko]

+1 from me

On Tue, Nov 10, 2015 at 6:38 PM, Stanislaw Bogatkin 
wrote:

> I think that it is excellent thought.
> +1
>
> On Tue, Nov 10, 2015 at 6:52 PM, Vladimir Kuklin 
> wrote:
>
>> Folks
>>
>> I wanted to raise awareness about one of the things I captured while
>> doing reviews recently - we are sacrificing quality to bugfixing and
>> feature development velocity, essentially moving from one heap to another -
>> from bugs/features to 'tech-debt' bugs.
>>
>> I understand that we all have deadlines and need to meet them. But,
>> folks, let's create the following policy:
>>
>> 1) do not introduce hacks/workarounds/kludges if it is possible.
>> 2) while fixing things if you have a hack/workaround/kludge that you need
>> to work with - think of removing it instead of enhancing and extending it.
>> If it is possible - fix it. Do not let our technical debt grow.
>> 3) if there is no way to avoid kludge addition/enhancing, if there is no
>> way to remove it - please, add a 'TODO/FIXME' line above it, so that we can
>> collect them in the future and fix them gradually.
>>
>> I suggest to add this requirement into code-review policy.
>>
>> What do you think about this?
>>
>> --
>> Yours Faithfully,
>> Vladimir Kuklin,
>> Fuel Library Tech Lead,
>> Mirantis, Inc.
>> +7 (495) 640-49-04
>> +7 (926) 702-39-68
>> Skype kuklinvv
>> 35bk3, Vorontsovskaya Str.
>> Moscow, Russia,
>> www.mirantis.com 
>> www.mirantis.ru
>> vkuk...@mirantis.com
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Change VIP address via API

[Aleksandr Didenko]

Hi,

Mike, that's exactly how you can use this VIPs allocation functionality.
Nailgun will save that VIP so it's not going to be auto-assigned to any
node and serialize it into astute.yaml (vip_name: IP). After that you can
get your VIP via Hiera and use it in your deployment scripts.

Guys, could you please clarify what's the purpose of 'node_roles' in VIP
description?

Regards,
Alex

On Fri, Nov 6, 2015 at 5:15 AM, Mike Scherbakov 
wrote:

> Is there a way to make it more generic, not "VIP" specific? Let's say I
> want to reserve address(-es) for something for whatever reason, and then I
> want to use them by some tricky way.
> More specifically, can we reserve IP address(-es) with some codename, and
> use it later?
> 12.12.12.12 - my-shared-ip
> 240.0.0.2 - my-multicast
> and then use them in puppet / whatever deployment code by $my-shared-ip,
> $my-multicast?
>
> Thanks,
>
> On Tue, Nov 3, 2015 at 8:49 AM Aleksey Kasatkin 
> wrote:
>
>> Folks,
>>
>> Here is a resume of our recent discussion:
>>
>> 1. Add new URLs for processing VIPs:
>>
>> /clusters//network_configuration/vips/ (GET)
>> /clusters//network_configuration/vips// (GET, PUT)
>>
>> where  is the id in ip_addrs table.
>> So, user can get all VIPS, get one VIP by id, change parameters (IP
>> address) for one VIP by its id.
>> More possibilities can be added later.
>>
>> Q. Any allocated IP could be accessible via these handlers, so now we can
>> restrict user to access VIPs only
>> and answer with some error to other ip_addrs ids.
>>
>> 2. Add current VIP meta into ip_addrs table.
>>
>> Create new field in ip_addrs table for placing VIP metadata there.
>> Current set of ip_addrs fields:
>> id (int),
>> network (FK),
>> node (FK),
>> ip_addr (string),
>> vip_type (string),
>> network_data (relation),
>> node_data (relation)
>>
>> Q. We could replace vip_type (it contains VIP name now) with vip_info.
>>
>> 3. Allocate VIPs on cluster creation and seek VIPs at all network changes.
>>
>> So, VIPs will be checked (via network roles descriptions) and
>> re-allocated in ip_addrs table
>> at these points:
>> a. create cluster
>> b. modify networks configuration
>> c. modify one network
>> d. modify network template
>> e. change nodes set for cluster
>> f. change node roles set on nodes
>> g. modify cluster attributes (change set of plugins)
>> h. modify release
>>
>> 4. Add 'manual' field into VIP meta to indicate whether it is
>> auto-allocated or not.
>>
>> So, whole VIP description may look like:
>> {
>> 'name': 'management'
>> 'network_role': 'mgmt/vip',
>> 'namespace': 'haproxy',
>> 'node_roles': ['controller'],
>> 'alias': 'management_vip',
>> 'manual': True,
>> }
>>
>> Example of current VIP description:
>>
>> https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/fixtures/openstack.yaml#L207
>>
>> Nailgun will re-allocate VIP address if 'manual' == False.
>>
>> 5. Q. what to do when the given address overlaps with the network from
>> another
>> environment? overlaps with the network of current environment which does
>> not match the
>> network role of the VIP?
>>
>> Use '--force' parameter to change it. PUT will fail otherwise.
>>
>>
>> Guys, please review this and share your comments here,
>>
>> Thanks,
>>
>>
>>
>> Aleksey Kasatkin
>>
>>
>> On Tue, Nov 3, 2015 at 10:47 AM, Aleksey Kasatkin > > wrote:
>>
>>> Igor,
>>>
>>> > For VIP allocation we should use POST request. It's ok to use PUT for
>>> setting (changing) IP address.
>>>
>>> My proposal is about setting IP addresses for VIPs only (auto and
>>> manual).
>>> No any other allocations.
>>> Do you propose to use POST for first-time IP allocation and PUT for IP
>>> re-allocation?
>>> Or use POST for adding entries to some new 'vips' table (so that all
>>> VIPs descriptions
>>> will be added there from network roles)?
>>>
>>> > We don't store network_role, namespace and node_roles within VIPs.
>>> > They are belonged to network roles. So how are you going to retrieve
>>> > them? Did you plan to make some changes to our data model? You know,
>>> > it's not a good idea to make connections between network roles and
>>> > VIPs each time your make a GET request to list them.
>>>
>>> It's our current format we use in API when VIPs are being retrieved.
>>> Do you propose to use different one for address allocation?
>>>
>>> > Should we return VIPs that aren't allocated, and if so - why? If they
>>> > would be just, you know, fetched from network roles - that's a bad
>>> > design. Each VIP should have an explicit entry in VIPs database table.
>>>
>>> I propose to return VIPs even w/o IP addresses to show user what VIPs he
>>> has
>>> so he can assign IP addresses to them. Yes, I supposed that the
>>> information
>>> will be retrieved from network roles as it is done now. Do you propose
>>> to create
>>> separate table for VIPs or extend ip_addrs table to 

Re: [openstack-dev] [Fuel][library] CI gate for regressions detection in deployment data

[Aleksandr Didenko]

Hi,

let me try to rephrase this a bit and Bogdan will correct me if I'm wrong
or missing something.

We have a set of top-scope manifests (called Fuel puppet tasks) that we use
for OpenStack deployment. We execute those tasks with "puppet apply". Each
task supposed to bring target system into some desired state, so puppet
compiles a catalog and applies it. So basically, puppet catalog = desired
system state.

So we can compile* catalogs for all top-scope manifests in master branch
and store those compiled* catalogs in fuel-library repo. Then for each
proposed patch CI will compare new catalogs with stored ones and print out
the difference if any. This will pretty much show what is going to be
changed in system configuration by proposed patch.

We were discussing such checks before several times, iirc, but we did not
have right tools to implement such thing before. Well, now we do :) I think
it could be quite useful even in non-voting mode.

* By saying compiled catalogs I don't mean actual/real puppet catalogs, I
mean sorted lists of all classes/resources with all parameters that we find
during puppet-rspec tests in our noop test framework, something like
standard puppet-rspec coverage. See example [0] for networks.pp task [1].

Regards,
Alex

[0] http://paste.openstack.org/show/477839/
[1]
https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/openstack-network/networks.pp


On Mon, Nov 2, 2015 at 5:35 PM, Bogdan Dobrelya 
wrote:

> Here is a docs update [0] for the patch [1] - which is rather a
> framework - being discussed here.
> Note, that the tool fuel_noop_tests.rb Dmitry Ilyin wrote became a Noop
> testing framework, which is Fuel specific. But the same approach may be
> used for any set of puppet modules and a composition layer manifests
> with a dataset of deployment parameters you may want it to be tracked
> against potential regressions.
>
> I believe we should think about how that Noop testing framework (and
> the deployment data checks under discussion as well) might benefit the
> puppet community.
>
> [1] https://review.openstack.org/240901
> [2] https://review.openstack.org/240015
>
> On 29.10.2015 15:24, Bogdan Dobrelya wrote:
> > Hello.
> > There are few types of a deployment regressions possible. When changing
> > a module version to be used from upstream (or internal module repo), for
> > example from Liberty to Mitaka. Or when changing the composition layer
> > (modular tasks in Fuel). Specifically, adding/removing/changing classes
> > and a class parameters.
> >
> > An example regression for swift deployment data [0]. Something was
> > changed unnoticed by existing noop tests and as a result
> > the swift data became being stored in root partition.
> >
> > Suggested per-commit based regressions detection [1] for deployment data
> > assumes to automatically detect if a class in a noop catalog run has
> > gained or lost a parameter or if it has been updated to another value by
> > a patch under test. Later, this check could even replace existing noop
> > tests, everything will be checked automatically, unless every deployment
> > scenario is covered by a corresponding template, which are represented
> > as YAML files [2] in Fuel.
> > Note: The tool [3] can help to get all deployment cases (-Y) and all
> > deployment tasks (-S) as well.
> >
> > I propose to review the patch [1], understand how it works (see tl;dr
> > section below) and to start using it ASAP. The earlier we commit the
> > "initial" data layer state, less regressions would pop up.
> >
> > (tl;dr)
> > The check should be done for every modular component (aka deployment
> > task). Data generated in the noop catalog run for all classes and
> > defines of a given deployment task should be verified against its
> > "acknowledged" (committed) state.
> > And fail the test gate, if changes has been found, like new parameter
> > with a defined value, removed a parameter, changed a parameter's value.
> >
> > In order to remove a regression, a patch author will have to add (and
> > reviewers should acknowledge) detected changes in the committed state of
> > the deployment data. This may be done manually, with a tool like [3] or
> > by a pre-commit hook, or even at the CI side!
> > The regression check should show the diff between committed state and a
> > new state proposed in a patch. Changed state should be *reviewed* and
> > accepted with a patch, to became a committed one. So the deployment data
> > will evolve with *only* approved changes. And those changes would be
> > very easy to be discovered for each patch under review process!
> > No more regressions, everyone happy.
> >
> > Examples:
> >
> > - A. A patch author removed the mpm_module parameter from the
> > composition layer (apache modular task). The test should fail with a
> >
> > Diff:
> >   @@ -90,7 +90,7 @@
> >  manage_user=> 'true',
> >  max_keepalive_requests => '100',
> 

Re: [openstack-dev] [Fuel][Plugins] Role for Fuel Master Node

[Aleksandr Didenko]

Hi,

please note that such tasks are executed inside 'mcollective' docker
container, not on the Fuel master host system.

Regards,
Alex

On Tue, Nov 3, 2015 at 10:41 PM, Igor Kalnitsky 
wrote:

> Hi Javeria,
>
> Try to use 'master' in 'role' field. Example:
>
> - role: 'master'
>   stage: pre_deployment
>   type: shell
>   parameters:
>   cmd: echo all > /tmp/plugin.all
>   timeout: 42
>
> Let me know if you need additional help.
>
> Thanks,
> Igor
>
> P.S: Since Fuel 7.0 it's recommended to use deployment_tasks.yaml
> instead of tasks.yaml. Please see Fuel Plugins wiki page for details.
>
> On Tue, Nov 3, 2015 at 10:26 PM, Javeria Khan 
> wrote:
> > Hey everyone,
> >
> > I've been working on a fuel plugin and for some reason just cant figure
> out
> > how to run a task on the fuel master node through the tasks.yaml. Is
> there
> > even a role for it?
> >
> > Something similar to what ansible does with localhost would work.
> >
> > Thanks,
> > Javeria
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Fuel-Library] Nominating Alex Schultz to Fuel-Library Core

[Aleksandr Didenko]

Hi,

+1 from me.

Regards,
Alex

On Wed, Sep 2, 2015 at 5:00 PM, Vladimir Kuklin 
wrote:

> +1 and also he is in US timezone, which  should help us cover the globe
> with Continuous Review process.
>
> On Wed, Sep 2, 2015 at 3:45 PM, Dmitry Borodaenko <
> dborodae...@mirantis.com> wrote:
>
>> Huge +1 from me, Alex has all the best qualities of a core reviewer:
>> great engineer, great communicator, diligent, and patient.
>>
>> On Wed, Sep 2, 2015 at 3:24 PM Jay Pipes  wrote:
>>
>>> I'm not a Fuel core or anything, but +1 from me. Alex has been very
>>> visible in the community and his work on librarian-puppet was a great
>>> step forward for the project.
>>>
>>> Best,
>>> -jay
>>>
>>> On 09/02/2015 04:31 AM, Sergii Golovatiuk wrote:
>>> > Hi,
>>> >
>>> > I would like to nominate Alex Schultz to Fuel-Library Core team. He’s
>>> > been doing a great job in writing patches. At the same time his reviews
>>> > are solid with comments for further improvements. He’s #3 reviewer and
>>> > #1 contributor with 46 commits for last 90 days [1]. Additionally, Alex
>>> > has been very active in IRC providing great ideas. His ‘librarian’
>>> > blueprint [3] made a big step towards to puppet community.
>>> >
>>> > Fuel Library, please vote with +1/-1 for approval/objection. Voting
>>> will
>>> > be open until September 9th. This will go forward after voting is
>>> closed
>>> > if there are no objections.
>>> >
>>> > Overall contribution:
>>> > [0] http://stackalytics.com/?user_id=alex-schultz
>>> > Fuel library contribution for last 90 days:
>>> > [1] http://stackalytics.com/report/contribution/fuel-library/90
>>> > List of reviews:
>>> > [2]
>>> >
>>> https://review.openstack.org/#/q/reviewer:%22Alex+Schultz%22+status:merged,n,z
>>> > ‘Librarian activities’ in mailing list:
>>> > [3]
>>> http://lists.openstack.org/pipermail/openstack-dev/2015-July/071058.html
>>> >
>>> > --
>>> > Best regards,
>>> > Sergii Golovatiuk,
>>> > Skype #golserge
>>> > IRC #holser
>>> >
>>> >
>>> >
>>> __
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com 
> www.mirantis.ru
> vkuk...@mirantis.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Code review process in Fuel and related issues

[Aleksandr Didenko]

Hi,

I'm all in for any formalization and automation of review process. The only
concern that I see here is about core reviewers involvement metrics. If we
succeed in reducing the load on core reviewers, it will mean that core
reviewers will do less code reviews. This could lead to core reviewer
demotion.

 - Contributor finds SME to review the code. Ideally, contributor can have
his/her peers to help with code review first. Contributor doesn’t bother
SME, if CI has -1 on a patch proposed

I like the idea with adding reviewers automatically based on MAINTAINERS
file. In such case we can drop this ^^ part of instruction. It would be
nice if Jenkins could add reviewers after CI +1, or we can use gerrit
dashboard for SMEs to not waste their time on review that has not yet
passed CI and does not have +1 from other reviewers.

Regards,
Alex


On Wed, Aug 19, 2015 at 11:31 AM, Mike Scherbakov mscherba...@mirantis.com
wrote:

 Hi all,
 let's discuss code review process in Fuel and what we can improve. For
 those who want to just have a quick context of this email, please check out
 presentation slides [5].

 ** Issues **
 Depending on a Fuel subproject, I'm aware of two buckets of issues with
 code review in Fuel:
 a) It is hard to get code reviewed and merged
 b) Quality of code review itself could be better

 First bucket:
 1) It is hard to find subject matter experts who can help and core
 reviewers for the area of code, especially if you are new to the project
 2) Contributor sometimes receives contradicting opinions from other
 reviewers, including cores
 3) Assigned / responsible core reviewer is needed for a feature in order
 to help in architectural negotiations, guiding through, landing the code
 into master
 4) Long wait time for getting code reviewed

 Quality-related items:
 5) Not thorough enough, full review in one shot. For example, reviewer can
 put -1 due to missed comma, but do not notice major gap in the code. It
 leads to many patch sets, and demotivation of contributors
 6) Some of the core reviewers decreased their involvement, and so number
 of reviews has dropped dramatically. However, they still occasionally merge
 code. I propose to remove these cores, and get them back if their
 involvement is increased back again (I very rarely merge code, but I'm one
 of those to be removed from cores). This is standard practice in OpenStack
 community as well, see Neutron as example [4, line 270].
 7) As a legacy of the past, we still have old core reviewers being able to
 merge code in all Fuel repos. All new cores have core rights only for
 single repo, which is their primary area of expertise. For example, core
 team size for fuel-library is adidenko + whole fuel-core group [7]. In
 fact, there are just 4 trusted or real core reviewers in fuel-library,
 not the whole fuel-core group.

 These problems are not new to OpenStack and open source in general. You
 can find discussions about same and similar issues in [1], [2], [3].


 ** Analysis of data **
 In order to understand what can be improved, I mined the data at first.
 Main source of information was stackalytics.com. Please take a look at
 few graphs on slides 4-7 [5], built based on data from stackalytics. Major
 conclusions from these graphs:
 1) Rather small number of core reviewers (in comparison with overall
 number of contributors) reviewing 40-60% of patch sets, depending on repo
 (40% fuel-library, 60% fuel-web). See slide #4.
 2) Load on core reviewers in Fuel team is higher in average, if you
 compare it with some other OpenStack projects. Average load on core
 reviewer across Nova, Keystone, Neutron and Cinder is 2.5 reviews a day. In
 Fuel though it is 3.6 for fuel-web and 4.6 for fuel-library. See slide #6.
 3) Statistics on how fast feedback on code proposed is provided:
 - fuel-library: 2095 total reviews in 30 days [13], 80 open reviews,
 average wait time for reviewer - 1d 1h [12]
 - fuel-web: 1789 total reviews in 30 days [14], 52 open reviews, average
 wait time for reviewer - 1d 17h [15]

 There is no need to have deep analysis on whether we have well defined
 areas of ownership in Fuel components or not: we don’t have it formally
 defined, and it’s not documented anywhere. So, finding a right core
 reviewer can be challenging task for a new contributor to Fuel, and this
 issue has to be addressed.


 ** Proposed solution **
 According to stackalytics, for the whole fuel-group we had 262 reviewers
 with 24 core reviewers for the past 180 days [19]. I think that these
 numbers can be considered as high enough in order to think about structure
 in which code review process would be transparent, understandable and
 scalable.

 Let’s first agree on the terminology which I’d like to use. It can take
 pages of precise definitions, however in this email thread I’d like to
 focus on code review process more, and hopefully high level description of
 roles would be enough for now.
 - Contributor: new contributor, who doesn’t work on 

Re: [openstack-dev] [Fuel] Aligning LP groups with real teams

[Aleksandr Didenko]

Hi,

just wanted to let you know that fuel-astute and fuel-provisioning groups
have been removed from LP.

 BTW, any chance we can somehow to reduce spam emails when some bug was
assigned to another team?

Igor, I'd recommend to setup email filters and lables. There are messages
like You received this bug notification because you are a member of Fuel
for Openstack, which is a bug assignee or You received this bug
notification because you are a member of Fuel Library Team, which is a bug
assignee. So you can direct such emails into different folders/labels.

Regards,
Alex

On Fri, Jul 17, 2015 at 4:26 PM, Igor Kalnitsky ikalnit...@mirantis.com
wrote:

 Hello,

 Here's my +2 on this. :)

 BTW, any chance we can somehow to reduce spam emails when some bug was
 assigned to another team? For instance, I see email notifications when
 bug's assigned to fuel-library.

 Thanks,
 Igor

 On Fri, Jul 17, 2015 at 4:16 PM, Tatyana Leontovich
 tleontov...@mirantis.com wrote:
  Hi,
 
  Alex vote +1 to use astute tag as well to get possibility identify issues
  related to astute in the most easy way.
 
  Regards,
  Tanya
 
  On Fri, Jul 17, 2015 at 3:59 PM, Aleksandr Didenko 
 adide...@mirantis.com
  wrote:
 
  Hi,
 
  as we decided on the recent Fuel weekly IRC meeting, we need to align LP
  fuel-* groups with our teams and bug confirmation queues/duties. We
 decided
  to start with fuel-astute [0] and fuel-provsioning [1] LP groups that
 have 2
  members each. So from now on please assign bugs about provisioning and
  astute to fuel-python [2] LP gorup and add 'ibp' tag for bugs about
  provisioning.
  Guys from fuel-python, please pay attention to bug tags. If you're the
 SME
  for 'ibp', please take a look at 'ibp' bugs first.
 
  Btw should we also use 'astute' tag for the same purpose?
 
  Also we need someone to delete  fuel-astute and fuel-provisioning groups
  from LP, if there are no objections.
 
  Regards,
  Alex
 
  [0] https://launchpad.net/~fuel-astute/+members#active
  [1] https://launchpad.net/~fuel-provisioning/+members#active
  [2] https://launchpad.net/~fuel-python/+members#active
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][CI] CI on commit message

[Aleksandr Didenko]

Hi,

 I think that checking commit message compliance to commit message
guidelines (for example ending the first line with dot) is part of CI jobs,
and they will vote -1 if message is wrongly structured.


Maciej, we don't have such checks at the moment. You can craft any commit
message you want and it will not cause any problems with CI. I think,
reviewers could do the job on commit message verification and they already
do this :)

Regards,
Alex


On Wed, Jul 29, 2015 at 11:23 AM, Sergey Vasilenko svasile...@mirantis.com
wrote:

 -1 to Maciej
 +1 to Sergii



 /sv


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] [FFE] FF Exception request for Deploy nova-compute (VCDriver) feature

[Aleksandr Didenko]

Hi,

I agree with Sergii, the patch had some problems only with tests which are
already resolved. So I vote for FFE.

P.S. We've just merged it.

Regards,
Alex

On Mon, Jul 27, 2015 at 3:30 PM, Sergii Golovatiuk sgolovat...@mirantis.com
 wrote:

 Hi,

 I have checked the code. After fixing tests, this patch maybe included to
 FFE as it has minimal impact on core functionality. +1 for FFE for
 https://review.openstack.org/#/c/196114/

 --
 Best regards,
 Sergii Golovatiuk,
 Skype #golserge
 IRC #holser

 On Mon, Jul 27, 2015 at 1:38 PM, Vladimir Kuklin vkuk...@mirantis.com
 wrote:

 There is a slight change needed, e.g. fixing of noop tests. Then we can
 merge it and accept for FFE, I think.

 On Fri, Jul 24, 2015 at 1:34 PM, Andrian Noga an...@mirantis.com wrote:

 Colleagues,
 actually, i'm tottaly agree with Mike. We can merge
 https://review.openstack.org/#/c/196114/ w/o additional Ceilometer
 support (will be moved to next release). So if we merge it today we dont
 need FFE for this feature.


 Regards,
 Andrian

 On Fri, Jul 24, 2015 at 1:18 AM, Mike Scherbakov 
 mscherba...@mirantis.com wrote:

 Since we are in FF state already, I'd like to have urgent estimate from
 one of fuel-library cores:
 - holser
 - alex_didenko
 - aglarendil
 - bogdando

 aglarendil is on vacation though. Guys, please take a look at
 https://review.openstack.org/#/c/196114/ - can we accept it as
 exception? Seems to be good to go...

 I still think that additional Ceilometer support should be moved to the
 next release.

 Thanks,

 On Thu, Jul 23, 2015 at 1:56 PM Mike Scherbakov 
 mscherba...@mirantis.com wrote:

 Hi Andrian,
 this is High priority blueprint [1] for 7.0 timeframe. It seems we
 still didn't merge the main part [2], and need FF exception for additional
 stuff.

 The question is about quality. If we focus on enhancements, then we
 don't focus on bugs. Which whether means to deliver work with lower 
 quality
 of slip the release.

 My opinion is rather don't give FF exception in this case, and don't
 have Ceilometer support for this new feature.

 [1] https://blueprints.launchpad.net/fuel/+spec/compute-vmware-role
 [2] https://review.openstack.org/#/c/196114/

 On Thu, Jul 23, 2015 at 1:39 PM Andrian Noga an...@mirantis.com
 wrote:

 Hi,

 The patch patch for fuel-library
 https://review.openstack.org/#/c/196114/  that implements
 'compute-vwmare' role (https://mirantis.jira.com/browse/PROD-627) 
 requires
 additional work to do (ceilometer support.), but as far as I can see it
 doesn't affect any other parts of the product.

 We plan to implement it in 3 working days (2 for implementation, 1
 day for writing system test and test runs), it should not be hard since 
 we
 already support ceilometer compute agent deployment on controller
 nodes.

 We need 1 DevOps engineer and 1 QA engineer to be engaged for this
 work.

 So I think it's ok to accept this feature as an exception for feature
 freeze.

 Regards,
 Andrian Noga
 Project manager
 Partner Centric Engineering
 Mirantis, Inc

 Mob.phone: +38 (063) 966-21-24

 Email: an...@mirantis.com
 Skype: bigfoot_ua

 --
 Mike Scherbakov
 #mihgen

 --
 Mike Scherbakov
 #mihgen




 --
 --
 Regards,
 Andrian
 Mirantis, Inc

 Mob.phone: +38 (063) 966-21-24
 Email: an...@mirantis.com
 Skype: bigfoot_ua


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 35bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] FF Exception request for Fernet tokens support

[Aleksandr Didenko]

Hi,

we were not able to get a working deployment with fernet token support
patches, mostly due to issues with keys generation and deployment
mechanism. I've also spend some time debugging problems with this and I
think it's too risky to land it in 7.0. So I vote for postponing it till
8.0.

Regards,
Alex

On Fri, Jul 24, 2015 at 10:17 AM, Bogdan Dobrelya bdobre...@mirantis.com
wrote:

  Fuel Library team, I expect your immediate reply here.
 
  I'd like upgrades team to take a look at this one, as well as at the one
  which moves Keystone under Apache, in order to check that there are no
  issues here.
 
  -1 from me for this time in the cycle. I'm concerned about:
 
 1. I don't see any reference to blueprint or bug which explains (with
 measurements) why we need this change in reference architecture, and
 what
 are the thoughts about it in puppet-openstack, and OpenStack
 Keystone. We
 need to get datapoints, and point to them. Just knowing that Keystone
 team
 implemented support for it doesn't yet mean that we need to rush in
 enabling this.
 2. It is quite noticeable change, not a simple enhancement. I reviewed
 the patch, there are questions raised.
 3. It doesn't pass CI, and I don't have information on risks
 associated,
 and additional effort required to get this done (how long would it
 take to
 get it done)
 4. This feature increases complexity of reference architecture. Now
 I'd
 like every complexity increase to be optional. I have feedback from
 the
 field, that our prescriptive architecture just doesn't fit users'
 needs,
 and it is so painful to decouple then what is needed vs what is not.
 Let's
 start extending stuff with an easy switch, being propagated from Fuel
 Settings. Is it possible to do? How complex would it be?
 
  If we get answers for all of this, and decide that we still want the
  feature, then it would be great to have it. I just don't feel that it's
  right timing anymore - we entered FF.
 
  Thanks,

 I vote -1 for the same reasons. Besides that, this feature seems already
 partially supported in puppet openstack upstream [0], hence should be
 developed and finished upstream first. Even if it wasn't at all - we
 should follow our contribution rules and do not develop features like
 Fernet tokens or v3 API in our forks.

 So, the next steps as I see them are:

 1) Freeze feature in fuel
 2) Submit a spec to openstack puppet to make the feature completed
 (address revocation, expiration, rotation of the fernet tokens). This
 also seems related to the already existing blueprint for fuel [1] and
 the mail thread [2]
 3) Implement the feature upstream
 4) Backport this to fuel fork in 8.0, or consume it upstream
 directly in the case the related blueprint [3] will be already
 implemented by that time.

 [0] https://review.openstack.org/185441
 [1] https://blueprints.launchpad.net/fuel/+spec/fernet-tokens-support
 [2]
 http://lists.openstack.org/pipermail/openstack-dev/2015-July/069744.html
 [3] https://blueprints.launchpad.net/fuel/+spec/fuel-puppet-librarian

 --
 Best regards,
 Bogdan Dobrelya,
 Irc #bogdando

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Switching keystone to apache wsgi

[Aleksandr Didenko]

Hi,

guys, we're about to switch keystone to apache wsgi by merging [0]. Just
wanted to make sure everyone is aware of this change.
If you have any questions or concerns let's discuss them in this thread.

Regards,
Alex

[0] https://review.openstack.org/#/c/204111/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Get rid of fuelmenu

[Aleksandr Didenko]

Hi,

 Am I alone in thinking it's not the best use of our development
 resources to throw it away and replace it with a text file that is
 edited by hand?

Nope. I'm with you :)

 Many people prefer vim UX instead of wandering through the semi-graphical
interface.

Are those ppl developers or end-users/customers?

Regards,
Alex


On Thu, Jul 23, 2015 at 5:26 PM, Oleg Gelbukh ogelb...@mirantis.com wrote:

 Unless I am mistaken, it is possible to set most of the parameters
 supported by Fuel menu as kernel boot parameters. Isn't it sufficient
 replacement for fuelmenu for dev's purposes?

 -Oleg

 On Thu, Jul 23, 2015 at 4:05 PM, Matthew Mosesohn mmoses...@mirantis.com
 wrote:

 How much effort are we spending? I'm not so sure it's a major development
 drain.

 Since Fuel 6.0 dev cycle (Sept 2014) until now there have been 34
 commits into Fuelmenu:
 * New features/functionality: 12
 * Bugfix: 15
 * Other: 7 (version bumps, and commits without bug ID)

 Across 3 releases, that's only ~11 commits per release. We've added
 features like generating random passwords for services, warnings about
 setting credentials apart from the default, adding a hook for CI for
 testing custom manifests on Fuel Master, and duplicate IP address
 checks.

 These improved user experience. If you take it away and replace it
 with a config file with basic validation, we will see users fail to
 deploy due to things that Fuelmenu already checks easily. Imagine
 you're an existing user of Fuel and suddenly you install the newest
 version of Fuel and see a large configuration file which you have to
 set by hand. Here's a relic of what users used to have to configure by
 hand:

 https://github.com/stackforge/fuel-library/blob/b015ed975b58dddff3b8da0ce34d9a638c22d032/deployment/puppet/openstack/examples/site_simple.pp

 Am I alone in thinking it's not the best use of our development
 resources to throw it away and replace it with a text file that is
 edited by hand?

 On Thu, Jul 23, 2015 at 3:33 PM, Igor Kalnitsky ikalnit...@mirantis.com
 wrote:
  Hello,
 
  Here's my 2 cents on it.
 
  I think the effort we put to support fuelmenu doesn't worth it. I used
  to deploy fuel too often in previous release, and I never used
  features of fuelmenu? Why? Because I prefer to apply changes on
  already deployed node. Moreover, I don't like that users are prompted
  with fuelmenu by default. I want to deploy fuel automatically, without
  any manual actions (though it's another topic).
 
  I'm agree with Vladimir, vim + config files are enough, since Fuel is
  not a product for housewives. It's a product for those who do not
  hesitate to use Vim for soft configuration.
 
  Thanks,
  Igor
 
 
 
  On Thu, Jul 23, 2015 at 2:27 PM, Matthew Mosesohn
  mmoses...@mirantis.com wrote:
  We had that before and had very poor validation. Removing fuelmenu
  would make the experience quite manual and prone to errors.
 
  This topic comes up once a year only from Fuel Python developers
  because they rarely use it and no dev cycles have been invested in
  improving it.
 
  The actual Fuel deployers use it and appreciate its validation and
  wish to extend it.
 
  I'd like to hear more feedback.
 
  On Thu, Jul 23, 2015 at 2:23 PM, Vladimir Kozhukalov
  vkozhuka...@mirantis.com wrote:
  Dear colleagues,
 
  What do you think of getting rid of fuelmenu and substituting it with
  thoroughly commented text file + some validation + vim? The major pro
 of
  this is that text file is easier to extend and edit. Many people
 prefer vim
  UX instead of wandering through the semi-graphical interface. And it
 is not
  so hard to implement syntax and logic checking for the text file.
 
  Please give your opinions about this.
 
  Vladimir Kozhukalov
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack 

Re: [openstack-dev] [Fuel][Plugins] Hiera nodes issue for 6.1 plugins

[Aleksandr Didenko]

Hi,

I think we should just fix the bug to make nodes.yaml match with the data
in astute.yaml. Because 'nodes' Hiera key is also used for /etc/hosts
update. I've raised bug priority to high.

Regards,
Alex

On Wed, Jul 22, 2015 at 2:42 PM, Irina Povolotskaya 
ipovolotsk...@mirantis.com wrote:

 Hi to all,

 Swann Croiset reported a bug on Hiera nodes [1].
 This issue affects several plugins so far.

 In 6.1, there is no workaround.
 In 7.0,  there should be a new structure for networks_metadata;
 this means, there will be ready-to-go puppet functions to get the data.

 Unfortunately, it impacts UX greatly.

 Thanks.


 [1] https://bugs.launchpad.net/fuel/+bug/1476957

 --
 Best regards,

 Irina

 *Business Analyst*

 *Partner Enablement*

 *skype: ira_live*







 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] librarian-puppet integration, need help with build tasks for fuel-library

[Aleksandr Didenko]

I believe build_repo function is the best way to do this [0]. So for
fuel-library we'll need to run a shell script right from the repo before
'touch $$@'. We can make it either conditional ( test -f
./path/additional_build_script.sh  bash ./path/additional_build_script.sh
) or as additional parameter to function and add it in fuel-library call [1]

Regards,
Alex

[0] https://github.com/stackforge/fuel-main/blob/master/repos.mk#L16-L37
[1] https://github.com/stackforge/fuel-main/blob/master/repos.mk#L45


On Fri, Jul 17, 2015 at 2:37 PM, Alex Schultz aschu...@mirantis.com wrote:

 Hey Alex,

 On Jul 17, 2015 4:32 AM, Aleksandr Didenko adide...@mirantis.com
 wrote:
 
  Hi,
 
  I think that we should provide a separate script that will fetch the
 upstream modules into fuel-library/deployment/puppet/ directory. It will
 allow us to have everything in a single place and use this script in ISO
 build process and CI jobs.
 

 Right. That is what I'm going for. The issue I need help with is the best
 way to execute this as part of the build process.  From what i understand
 of the build process is that we are using git archive for all pieces so I'm
 not sure how to wedge in an extra script execution to do the module fetch.
 The creation of the script isn't the issue, the issue is how can I properly
 run it as part of the build process.


  Regards,
  Alex
 

 Thanks,
 -Alex

  On Thu, Jul 16, 2015 at 11:17 PM, Alex Schultz aschu...@mirantis.com
 wrote:
 
  Hello everyone,
 
  I have committed the initial configuration required to start leveraging
 librarian-puppet as part of the way we pull in upstream puppet modules[0].
 Additionally, I have also committed a change that would pull in the
 openstack-ironic module[1].  The one piece that is missing from this being
 a complete solution is the ability to run librarian-puppet as part of our
 build process for the fuel-library.  I've looked into the fuel-main build
 scripts and I think it's over my head to figure this out just by looking.
 Can anyone explain to me or assist me in how I could go about modifying the
 existing build system to be able to run librarian-puppet to prepare the
 source for the package?  In my initial investigation, it looks like it
 would be a modification of the fuel-main/packages/module.mk[3] file.  I
 basically need to do the prepare_library[3] function from the 202763
 review[0] after we've pulled all the sources together to fetch the upstream
 modules.
 
 
  Thanks,
  -Alex
 
  [0] https://review.openstack.org/202763
  [1] https://review.openstack.org/202767
  [2]
 https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82
  [3]
 https://review.openstack.org/#/c/202763/1/utils/jenkins/fuel_noop_tests.rb
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Aligning LP groups with real teams

[Aleksandr Didenko]

Hi,

as we decided on the recent Fuel weekly IRC meeting, we need to align LP
fuel-* groups with our teams and bug confirmation queues/duties. We decided
to start with fuel-astute [0] and fuel-provsioning [1] LP groups that have
2 members each. So from now on please assign bugs about provisioning and
astute to fuel-python [2] LP gorup and add 'ibp' tag for bugs about
provisioning.
Guys from fuel-python, please pay attention to bug tags. If you're the SME
for 'ibp', please take a look at 'ibp' bugs first.

Btw should we also use 'astute' tag for the same purpose?

Also we need someone to delete  fuel-astute and fuel-provisioning groups
from LP, if there are no objections.

Regards,
Alex

[0] https://launchpad.net/~fuel-astute/+members#active
[1] https://launchpad.net/~fuel-provisioning/+members#active
[2] https://launchpad.net/~fuel-python/+members#active
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] librarian-puppet integration, need help with build tasks for fuel-library

[Aleksandr Didenko]

Hi,

I think that we should provide a separate script that will fetch the
upstream modules into fuel-library/deployment/puppet/ directory. It will
allow us to have everything in a single place and use this script in ISO
build process and CI jobs.

Regards,
Alex

On Thu, Jul 16, 2015 at 11:17 PM, Alex Schultz aschu...@mirantis.com
wrote:

 Hello everyone,

 I have committed the initial configuration required to start leveraging
 librarian-puppet as part of the way we pull in upstream puppet modules[0].
 Additionally, I have also committed a change that would pull in the
 openstack-ironic module[1].  The one piece that is missing from this being
 a complete solution is the ability to run librarian-puppet as part of our
 build process for the fuel-library.  I've looked into the fuel-main build
 scripts and I think it's over my head to figure this out just by looking.
 Can anyone explain to me or assist me in how I could go about modifying the
 existing build system to be able to run librarian-puppet to prepare the
 source for the package?  In my initial investigation, it looks like it
 would be a modification of the fuel-main/packages/module.mk[3] file.  I
 basically need to do the prepare_library[3] function from the 202763
 review[0] after we've pulled all the sources together to fetch the upstream
 modules.


 Thanks,
 -Alex

 [0] https://review.openstack.org/202763
 [1] https://review.openstack.org/202767
 [2]
 https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82
 [3]
 https://review.openstack.org/#/c/202763/1/utils/jenkins/fuel_noop_tests.rb

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Fuel-library] Using librarian-puppet to manage upstream fuel-library modules

[Aleksandr Didenko]

Hi,

guys, what if we simplify things a bit? All we need is:

   1. Remove all the community modules from fuel-library.
   2. Create 'Puppetfile' with list of community modules and their versions
   that we currently use.
   3. Make sure all our customizations are proposed to the upstream modules
   (via gerrit or github pull-requests).
   4. Create a separate file with list of patches for each module we need
   to cherry-pick (we need to support gerrit reviews and github pull-requests).
   5. Update 'make iso' scripts:
  1. Make them use 'r10k' (or other tool) to download upstream modules
  based on 'Puppetfile'
  2. Iterate over list of patches for each module and cherry-pick them
  (just like we do for custom ISO build. I'm not sure if librarian provides
  such possibility)

Eventually, when all the functionality we rely on is accepted in upstream
modules, we'll get rid of file with list of patches for modules. But
meanwhile it should be much easier to manage modules and customization in
such way.

Regards,

Alex



On Fri, Jul 10, 2015 at 5:25 PM, Alex Schultz aschu...@mirantis.com wrote:

 Done. Sorry about that.

 -Alex

 On Fri, Jul 10, 2015 at 9:22 AM, Simon Pasquier spasqu...@mirantis.com
 wrote:

 Alex, could you enable the comments for all on your document?
 Thanks!
 Simon

 On Thu, Jul 9, 2015 at 11:07 AM, Bogdan Dobrelya bdobre...@mirantis.com
 wrote:

  Hello everyone,
 
  I took some time this morning to write out a document[0] that outlines
  one possible ways for us to manage our upstream modules in a more
  consistent fashion. I know we've had a few emails bouncing around
  lately around this topic of our use of upstream modules and how can we
  improve this. I thought I would throw out my idea of leveraging
  librarian-puppet to manage the upstream modules within our
  fuel-library repository. Ideally, all upstream modules should come
  from upstream sources and be removed from the fuel-library itself.
  Unfortunately because of the way our repository sits today, this is a
  very large undertaking and we do not currently have a way to manage
  the inclusion of the modules in an automated way. I believe this is
  where librarian-puppet can come in handy and provide a way to manage
  the modules. Please take a look at my document[0] and let me know if
  there are any questions.
 
  Thanks,
  -Alex
 
  [0]
 https://docs.google.com/document/d/13aK1QOujp2leuHmbGMwNeZIRDr1bFgJi88nxE642xLA/edit?usp=sharing

 The document is great, Alex!
 I'm fully support the idea to start adapting fuel-library by
 the suggested scheme. The monitoring feature of ibrarian looks not
 intrusive and we have no blockers to start using the librarian just
 immediately.

 --
 Best regards,
 Bogdan Dobrelya,
 Irc #bogdando


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Fuel-library] Using librarian-puppet to manage upstream fuel-library modules

[Aleksandr Didenko]

Hi,

just wanted to mention another tool to work with 'Puppetfile' - r10k:

https://github.com/puppetlabs/r10k/blob/master/doc/puppetfile.mkd

Regards,
Alex

On Wed, Jun 24, 2015 at 11:04 PM, Paul Belanger pabelan...@redhat.com
wrote:

 On 06/23/2015 01:51 PM, Alex Schultz wrote:

 Hello everyone,

 I took some time this morning to write out a document[0] that outlines
 one possible ways for us to manage our upstream modules in a more
 consistent fashion. I know we've had a few emails bouncing around
 lately around this topic of our use of upstream modules and how can we
 improve this. I thought I would throw out my idea of leveraging
 librarian-puppet to manage the upstream modules within our
 fuel-library repository. Ideally, all upstream modules should come
 from upstream sources and be removed from the fuel-library itself.
 Unfortunately because of the way our repository sits today, this is a
 very large undertaking and we do not currently have a way to manage
 the inclusion of the modules in an automated way. I believe this is
 where librarian-puppet can come in handy and provide a way to manage
 the modules. Please take a look at my document[0] and let me know if
 there are any questions.

  I'd suggest looking at librarian-puppet-simple over librarian-puppet.  I
 found the dependency management terrible with librarian-puppet.  The more
 complex your puppet dependencies became, the longer the tooling took to run.

 Mine you this was a few years ago.

 PB



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Python code in fuel-library

[Aleksandr Didenko]

Hi,

I agree that we need a better testing for python tasks/code. There should
be no problems adding py.test tests into fuel-library CI, we already have
one [1] up and running. So I'm all in and ready to help with such testing
implementation.

[1] https://fuel-jenkins.mirantis.com/job/fuellib_tasks_graph_check/

Regards,
Aleksandr

On Wed, Feb 18, 2015 at 4:02 PM, Vladimir Kuklin vkuk...@mirantis.com
wrote:

 Hi, Seb

 Very fair point, thank you. We need to add this to our jobs for unittests
 run and syntax check. I am adding Aleksandr Didenko into the loop as he is
 currently working on the similar task.

 On Wed, Feb 18, 2015 at 4:53 PM, Jay Pipes jaypi...@gmail.com wrote:

 On 02/18/2015 04:57 AM, Sebastian Kalinowski wrote:

 Hello Fuelers,

 There is more and more Python code appearing in fuel-library [1] that is
 used in our Puppet manifests. Now, with introduction of Granular
 Deployment feature it could appear more often as
 writing some tasks as a Python script is a nice option.

 First problem that I see is that in some cases this code is getting
 merged without a positive review from a Python developer from Fuel team.
 My proposition of the solution is simple:
 fuel-library core reviewers shouldn't merge such code if there is no a
 +1 from a Python developer from fuel-core group [2].

 Second problem is that there are no automatic tests for this code.
 Testing it manually and by running deployment when that code is used is
 not enough since those scripts could be quite large and complicated and
 some of them are executed in specific situations so it is hard for
 reviewers to check how they will work.
 In fuel-library we already have tests for Puppet modules: [3].
 I suggest that we should introduce similar checks for Python code:
   - there will be one global 'test-requirements.txt' file (if there will
 be a need to, we could introduce more granular split, like per module)
   - py.test [4] will be used as a test runner
   - (optional, but advised) flake8+hacking checks [5] (could be limited
 to just run flake8/pyflakes checks)

 Looking forward to your opinions on those two issues.


 Hi Seba,

 All those suggestions look fine to me. I'd also add to improve the
 documentation on how to write and run Python tests to help out those
 developers who are not as familiar with Python as Ruby or other languages.

 Best,
 -jay

 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
 unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 45bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel][Fuel-Library] Manifests and tasks for granular deployment

[Aleksandr Didenko]

Hi,
General info
Detailed documentation is available in the latest spec [1] (see [2] for
HTML format). You can also check blue-print [3]  and etherpad how-to [4]
for more info.

Fuel-library granularization status
As you may know, we're using granular task based deployment in master
branch already. Currently we have a set of separate manifests that are
applied by puppet as separate tasks, they are stored here [5].

According to our implementation plan (step #2, see spec for details) we
moved node roles (controller, compute, cinder, ceph-osd, mongo,
zabbix-server) into separate tasks. So every role is deployed by its own
separate puppet manifest applied as Fuel task.

We're also going to remove the following files shortly as we no longer
use/need them:

deployment/puppet/osnailyfacter/examples/site.pp
deployment/puppet/osnailyfacter/manifests/cluster_ha.pp
deployment/puppet/osnailyfacter/manifests/cluster_simple.pp
deployment/puppet/osnailyfacter/modular/legacy.pp

So here are few examples, just to illustrate current deployment process:

1) Controller (primary and non-primary) role:
puppet apply /etc/puppet/modules/osnailyfacter/modular/hiera.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/globals.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/logging.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/netconfig.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/firewall.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/hosts.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/tools.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/controller.pp

2) Compute role:
puppet apply /etc/puppet/modules/osnailyfacter/modular/hiera.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/globals.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/logging.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/netconfig.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/firewall.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/hosts.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/tools.pp
puppet apply /etc/puppet/modules/osnailyfacter/modular/compute.pp

Hiera
Also we're switching from parseyaml() function and global $::fuel_settings
hash to Hiera. All the configuration data should be pulled into manifests
via hiera() only.  Hiera is configured by the first two tasks:

/etc/puppet/modules/osnailyfacter/modular/hiera.pp - configures Hiera
/etc/puppet/modules/osnailyfacter/modular/globals.pp - creates
/etc/hiera/globals.yaml file with node-specific calculated data that should
help to avoid code duplication in manifests

For example: globals.yaml contains *internal_address* for the node and you
can pull it via hiera:

root@node-1:~# hiera internal_address
192.168.0.3

Without globals.yaml you would have to search for internal_address in nodes
hash in astute.yaml. You can see current modular manifests for more
examples.

CI updates
We also have some changes in fuel-library CI related to granular
deployment. Since our deployment process depends on tasks, which are
shipped with fuel-library repo, we've added new CI test job [6] that runs
basic schema validation and makes sure we have acyclic tasks graph.


[1] https://review.openstack.org/147591
[2]
http://docs-draft.openstack.org/91/147591/13/check/gate-fuel-specs-docs/e58793b//doc/build/html/specs/6.1/fuel-library-modularization.html
[3] https://blueprints.launchpad.net/fuel/+spec/fuel-library-modularization
[4] https://etherpad.openstack.org/p/fuel-library-modularization
[5]
https://github.com/stackforge/fuel-library/tree/master/deployment/puppet/osnailyfacter/modular
[6] https://fuel-jenkins.mirantis.com/job/fuellib_tasks_graph_check

--
Regards,
Aleksandr Didenko
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] removing single mode

[Aleksandr Didenko]

What do you guys think about switching CentOS CI job [1] to HA with single
controller (1 controller + 1 or 2 computes)? Just to verify that our
replacement of Simple mode works fine.

[1]
https://fuel-jenkins.mirantis.com/job/master.fuel-library.centos.ha_nova_vlan/

On Fri, Jan 30, 2015 at 10:54 AM, Mike Scherbakov mscherba...@mirantis.com
wrote:

 Thanks Igor for the quick turn over, excellent!

 On Fri, Jan 30, 2015 at 1:19 AM, Igor Belikov ibeli...@mirantis.com
 wrote:

 Folks,

 Changes in CI jobs have been made, for master branch of fuel-library we
 are running CentOS HA + Nova VLAN and Ubuntu HA + Neutron VLAN .
 Job naming schema has also been changed, so now it includes actual
 testgroup. Current links for master branch CI jobs are [1] and [2], all
 other jobs can be found here[3] or will show up in your gerrit reviews.
 ISO and environments have been updated to the latest ones.

 [1]
 https://fuel-jenkins.mirantis.com/job/master.fuel-library.centos.ha_nova_vlan/
 [2]
 https://fuel-jenkins.mirantis.com/job/master.fuel-library.ubuntu.ha_neutron_vlan/
 [3]https://fuel-jenkins.mirantis.com
 --
 Igor Belikov
 Fuel DevOps
 ibeli...@mirantis.com





 On 29 Jan 2015, at 13:42, Aleksandr Didenko adide...@mirantis.com
 wrote:

 Mike,

  Any objections / additional suggestions?

 no objections from me, and it's already covered by LP 1415116 bug [1]

 [1] https://bugs.launchpad.net/fuel/+bug/1415116

 On Wed, Jan 28, 2015 at 6:42 PM, Mike Scherbakov 
 mscherba...@mirantis.com wrote:

 Folks,
 one of the things we should not forget about - is out Fuel CI gating
 jobs/tests. [1], [2].

 One of them is actually runs simple mode. Unfortunately, I don't see
 details about tests ran for [1], [2], but I'm pretty sure it's same set as
 [3], [4].

 I suggest to change tests. First of all, we need to get rid of simple
 runs (since we are deprecating it), and second - I'd like us to run Ubuntu
 HA + Neutron VLAN for one of the tests.

 Any objections / additional suggestions?

 [1]
 https://fuel-jenkins.mirantis.com/job/master_fuellib_review_systest_centos/
 [2]
 https://fuel-jenkins.mirantis.com/job/master_fuellib_review_systest_ubuntu/
 [3]
 https://fuel-jenkins.mirantis.com/job/6_0_fuellib_review_systest_centos/
 [4]
 https://fuel-jenkins.mirantis.com/job/6_0_fuellib_review_systest_ubuntu/

 On Wed, Jan 28, 2015 at 2:28 PM, Sergey Vasilenko 
 svasile...@mirantis.com wrote:

 +1 to replace simple to HA with one controller

 /sv


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Mike Scherbakov
 #mihgen



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org
 ?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Mike Scherbakov
 #mihgen


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] removing single mode

[Aleksandr Didenko]

Mike,

 Any objections / additional suggestions?

no objections from me, and it's already covered by LP 1415116 bug [1]

[1] https://bugs.launchpad.net/fuel/+bug/1415116

On Wed, Jan 28, 2015 at 6:42 PM, Mike Scherbakov mscherba...@mirantis.com
wrote:

 Folks,
 one of the things we should not forget about - is out Fuel CI gating
 jobs/tests. [1], [2].

 One of them is actually runs simple mode. Unfortunately, I don't see
 details about tests ran for [1], [2], but I'm pretty sure it's same set as
 [3], [4].

 I suggest to change tests. First of all, we need to get rid of simple runs
 (since we are deprecating it), and second - I'd like us to run Ubuntu HA +
 Neutron VLAN for one of the tests.

 Any objections / additional suggestions?

 [1]
 https://fuel-jenkins.mirantis.com/job/master_fuellib_review_systest_centos/
 [2]
 https://fuel-jenkins.mirantis.com/job/master_fuellib_review_systest_ubuntu/
 [3]
 https://fuel-jenkins.mirantis.com/job/6_0_fuellib_review_systest_centos/
 [4]
 https://fuel-jenkins.mirantis.com/job/6_0_fuellib_review_systest_ubuntu/

 On Wed, Jan 28, 2015 at 2:28 PM, Sergey Vasilenko svasile...@mirantis.com
  wrote:

 +1 to replace simple to HA with one controller

 /sv

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Mike Scherbakov
 #mihgen


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins][Orchestration] Unclear handling of primary-controler and controller roles

[Aleksandr Didenko]

Hi,

we definitely need such separation on orchestration layer.

 Is it possible to have significantly different sets of tasks for
controller and primary-controller?

Right now we already do different things on primary and secondary
controllers, but it's all conducted in the same manifest and controlled by
conditionals inside the manifest. So when we split our tasks into smaller
ones, we may want/need to separate them for primary and secondary
controllers.

 I wouldn't differentiate tasks for primary and other controllers.
Primary-controller logic should be controlled by task itself. That will
allow to have elegant and tiny task framework

Sergii, we still need this separation on the orchestration layer and, as
you know, our deployment process is based on it. Currently we already have
separate task groups for primary and secondary controller roles. So it will
be up to the task developer how to handle some particular task for
different roles: developer can write 2 different tasks (one for
'primary-controller' and the other one for 'controller'), or he can write
the same task for both groups and handle differences inside the task.

--
Regards,
Aleksandr Didenko


On Wed, Jan 28, 2015 at 11:25 AM, Dmitriy Shulyak dshul...@mirantis.com
wrote:

 But without this separation on orchestration layer, we are unable to
 differentiate between nodes.
 What i mean is - we need to run subset of tasks on primary first and then
 on all others, and we are using role as mapper
 to node identities (and this mechanism was hardcoded in nailgun for a long
 time).

 Lets say we have task A that is mapped to primary-controller and B that is
 mapped to secondary controller, task B requires task A.
 If there is no primary in mapping - we will execute task A on all
 controllers and then task B on all controllers.

 And how in such case deployment code will know that it should not execute
 commands in task A for secondary controllers and
 in task B on primary ?

 On Wed, Jan 28, 2015 at 10:44 AM, Sergii Golovatiuk 
 sgolovat...@mirantis.com wrote:

 Hi,

 *But with introduction of plugins and granular deployment, in my opinion,
 we need to be able*
 *to specify that task should run specifically on primary, or on
 secondaries. Alternative to this approach would be - always run task on all
 controllers, and let task itself to verify that it is  executed on primary
 or not.*

 I wouldn't differentiate tasks for primary and other controllers.
 Primary-controller logic should be controlled by task itself. That will
 allow to have elegant and tiny task framework ...

 --
 Best regards,
 Sergii Golovatiuk,
 Skype #golserge
 IRC #holser

 On Tue, Jan 27, 2015 at 11:35 PM, Dmitriy Shulyak dshul...@mirantis.com
 wrote:

 Hello all,

 You may know that for deployment configuration we are serializing
 additional prefix for controller role (primary), with the goal of
 deployment order control (primary-controller always should be deployed
 before secondaries) and some condiions in fuel-library code.

 However, we cannot guarantee that primary controller will be always the
 same node, because it is not business of nailgun to control elections of
 primary. Essentially user should not rely on nailgun
 information to find primary, but we need to persist node elected as
 primary in first deployment
 to resolve orchestration issues (when new node added to cluster we
 should not mark it as primary).

 So we called primary-controller - internal role, which means that it
 is not exposed to users (or external developers).
 But with introduction of plugins and granular deployment, in my opinion,
 we need to be able
 to specify that task should run specifically on primary, or on
 secondaries. Alternative to this approach would be - always run task on all
 controllers, and let task itself to verify that it is  executed on primary
 or not.

 Is it possible to have significantly different sets of tasks for
 controller and primary-controller?
 And same goes for mongo, and i think we had primary for swift also.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List

Re: [openstack-dev] [Fuel] Distribution of keys for environments

[Aleksandr Didenko]

3rd option is about using rsyncd that we run under xinetd on primary
controller. And yes, the main concern here is security.

On Wed, Jan 28, 2015 at 6:04 PM, Stanislaw Bogatkin sbogat...@mirantis.com
wrote:

 Hi.
 I'm vote for second option, cause if we will want to implement some
 unified hierarchy (like Fuel as CA for keys on controllers for different
 env's) then it will fit better than other options. If we implement 3rd
 option then we will reinvent the wheel with SSL in future. Bare rsync as
 storage for private keys sounds pretty uncomfortable for me.

 On Wed, Jan 28, 2015 at 6:44 PM, Dmitriy Shulyak dshul...@mirantis.com
 wrote:

 Hi folks,

 I want to discuss the way we are working with generated keys for
 nova/ceph/mongo and something else.

 Right now we are generating keys on master itself, and then distributing
 them by mcollective
 transport to all nodes. As you may know we are in the process of making
 this process described as
 task.

 There is a couple of options:
 1. Expose keys in rsync server on master, in folder /etc/fuel/keys, and
 then copy them with rsync task (but it feels not very secure)
 2. Copy keys from /etc/fuel/keys on master, to /var/lib/astute on target
 nodes. It will require additional
 hook in astute, smth like copy_file, which will copy data from file on
 master and put it on the node.

 Also there is 3rd option to generate keys right on primary-controller and
 then distribute them on all other nodes, and i guess it will be
 responsibility of controller to store current keys that are valid for
 cluster. Alex please provide more details about 3rd approach.

 Maybe there is more options?



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] removing single mode

[Aleksandr Didenko]

Hi,

After starting implementing granular deployment we've faced a bunch of
issues that would make further development of this feature much more
complicated if we have to support both Simple and HA deployment modes. For
example: simple mode does not require cluster (corosync, pacemaker, vips,
etc), so we had to skip this task for Simple mode somehow - we can use
conditional tasks, or conditional manifests in our tasks, or create
separate task graphs for different deployment modes, etc - either way it's
pretty much doubling the amount of work for some parts of Fuel and our
development cycle.

At the moment, CI blocks us from further development of fuel-library
modularization BP [2] because we still use Simple mode in CI. So in order
to proceed with this BP we have two options:

1) remove Simple mode from CI/QA and thus drop it completely from Fuel
2) double our efforts to support both Simple and HA modes in granular
deployment

We have a BP about single-controller HA [1]. HA with single controller
works just fine at the moment. So if you want to test Fuel on a minimum set
of nodes, you can do this on 3 nodes (Fuel master, controller, compute),
just like with Simple mode before. I suppose, it's time to finally drop
support for Simple mode in Fuel :)

[1] https://blueprints.launchpad.net/fuel/+spec/single-controller-ha
[2] https://blueprints.launchpad.net/fuel/+spec/fuel-library-modularization

--
Regards,
Aleksandr Didenko


On Tue, Aug 26, 2014 at 9:25 AM, Mike Scherbakov mscherba...@mirantis.com
wrote:

 Definitely fuel spec is needed :)


 On Mon, Aug 25, 2014 at 8:45 PM, Evgeniy L e...@mirantis.com wrote:

 Hi Andrew,

 I have some comments regarding to you action items

  2) Removing simple mode from the ui and tests
  3) Removing simple mode support from nailgun (maybe we leave it) and
 cli

 We shouldn't do it, because nailgun should handle both versions of
 cluster.
 What we have to do here is to use openstack.yaml to keep all possible
 modes.
 For new release there will be only ha, to manage previous releases we have
 to create data migrations in nailgun to create the filed with modes i.e.
 multinode
 and ha.

 Also fixes for ui are required too, I think it mostly related to wizard,
 'mode' tab
 where use can chose ha or non ha cluster in case of new release there
 should
 be only ha, and in case of old releases there should be ha and multinode.

 Thanks,



  On Mon, Aug 25, 2014 at 8:19 PM, Andrew Woodward xar...@gmail.com
 wrote:

  Started a new thread so that we don't hijack the older thread.
  as


 Andrew, will you work on it in 6.0? What are remaining items there?
 Also, it might affect our tests - simple mode runs faster so we use it for
 smoke ISO test. Anastasia, please confirm that we can switch smoke to
 one-ha-controller model, or even drop smoke at all and use BVT only
 (running CentOS 3 HA controllers and same with Ubuntu).


 The primary reason that we haven't disabled single yet is was due to [0]
 where we where having problems adding additional controllers. With the
 changes to galera and rabbit clustering it appears that we ended up fixing
 it already.

 The remaining issues are:
 1) Ensuring we have good test coverage for the cases we expect to
 support [1]
 2) Removing simple mode from the ui and tests
 3) Removing simple mode support from nailgun (maybe we leave it) and cli
 4) Updating documentation

 [0] https://bugs.launchpad.net/fuel/+bug/1350266
 [1] https://bugs.launchpad.net/fuel/+bug/1350266/comments/7

 --
 Andrew
 Mirantis
 Ceph community

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Mike Scherbakov
 #mihgen


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] We lost some commits during upstream puppet manifests merge

[Aleksandr Didenko]

 Also I’m just wondering how do we keep upstream modules in our repo? They
are not submodules, so how is it organized?

Currently, we don't have any automatic tracking system for changes we apply
to the community/upstream modules, that could help us to re-apply them
during the sync. Only git or diff comparison between original module and
our copy. But that should not be a problem when we finish current sync and
switch to the new contribution workflow described in the doc, Vladimir has
mentioned in the initial email [1].

Also, in the nearest future we're planning to add unit tests (rake spec)
and puppet noop tests into our CI. I think we should combine noop tests
with regression testing by using 'rake spec'. But this time I mean RSpec
tests for puppet host, not for specific classes as I suggested in the
previous email. Such tests would compile a complete catalog using our
'site.pp' for specific astute.yaml settings and it will check that needed
puppet resources present in the catalog and have needed attributes. Here's
a draft - [2]. It checks catalog compilation for a controller node and runs
few checks for 'keystone' class and keystone cache driver settings.

Since all the test logic is outside of our puppet modules directory, it
won't be affected by any further upstream syncs or changes we apply in our
modules. So in case some commit removes anything critical that is covered
by regression/noop tests, then it will get '-1' from CI and attract our
attention :)

[1]
http://docs.mirantis.com/fuel-dev/develop/module_structure.html#contributing-to-existing-fuel-library-modules
[2] https://review.openstack.org/141022

Regards,
Aleksandr


On Fri, Nov 21, 2014 at 8:07 PM, Tomasz Napierala tnapier...@mirantis.com
wrote:


  On 21 Nov 2014, at 17:15, Aleksandr Didenko adide...@mirantis.com
 wrote:
 
  Hi,
 
  following our docs/workflow plus writing rspec tests for every new
 option we add/modify in our manifests could help with regressions. For
 example:
• we add new keystone config option in openstack::keystone class -
 keystone_config {'cache/backend': value = 'keystone.cache.memcache_pool';}
• we create new test for openstack::keystone class, something like
 this:
• should
 contain_keystone_config(cache/backend).with_value('keystone.cache.memcache_pool')
  So with such test, if for some reason we lose
 keystone_config(cache/backend) option, 'rake spec' would alert us about
 it right away and we'll get -1 from CI. Of course we should also
 implement 'rake spec' CI gate for this.
 
  But from the other hand, if someone changes option in manifests and
 updates rspec tests accordingly, then such commit will pass 'rake spec'
 test and we can still lose some specific option.
 
   We should speed up development of some modular testing framework that
 will check that corresponding change affects only particular pieces.
 
  Such test would not catch this particular regressions with
 keystone_config {'cache/backend': value =
 'keystone.cache.memcache_pool';}, because even with regression (i.e.
 dogpile backend) keystone was working OK. It has passed several BVTs and
 custom system tests, because 'dogpile' cache backend was working just fine
 while all memcached servers are up and running. So it looks like we need
 some kind of tests that will ensure that particular config options (or
 particular puppet resources) have some particular values (like backend =
 keystone.cache.memcache_pool in [cache] block of keystone.conf).
 
  So I would go with rspec testing for specific resources but I would
 write them in 'openstack' module. Those tests should check that needed
 (nova/cinder/keystone/glance)_config resources have needed values in the
 puppet catalog. Since we're not going to sync 'openstack' module with the
 upstream, such tests will remain intact until we change them, and they
 won't be affected by other modules sync/merge (keystone, cinder, nova, etc).

 I totally agree, but we need to remember to introduce tests in separate
 commits, otherwise loosing commit ID we would also lose tests ;)

 Also I’m just wondering how do we keep upstream modules in our repo? They
 are not submodules, so how is it organized?

 Regards,
 --
 Tomasz 'Zen' Napierala
 Sr. OpenStack Engineer
 tnapier...@mirantis.com







 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] fuel master monitoring

[Aleksandr Didenko]

Hi,

Dmitriy, first of all, monit can provide HTTP interface for communication -
so it's possible to poll that this interface to get info or even control
monit (stop/start/restart service, stop/start monitoring of a service,
etc). Secondly, you can configure different triggers in monit and set
appropriate actions, for example running some script - and that script can
do what ever you like. As for built-in possibility to send http requests -
I've never heard about it.

Regards,
Alex

On Thu, Nov 27, 2014 at 10:53 AM, Dmitriy Shulyak dshul...@mirantis.com
wrote:

 Is it possible to send http requests from monit, e.g for creating
 notifications?
 I scanned through the docs and found only alerts for sending mail,
 also where token (username/pass) for monit will be stored?

 Or maybe there is another plan? without any api interaction

 On Thu, Nov 27, 2014 at 9:39 AM, Przemyslaw Kaminski 
 pkamin...@mirantis.com wrote:

  This I didn't know. It's true in fact, I checked the manifests. Though
 monit is not deployed yet because of lack of packages in Fuel ISO. Anyways,
 I think the argument about using yet another monitoring service is now
 rendered invalid.

 So +1 for monit? :)

 P.


 On 11/26/2014 05:55 PM, Sergii Golovatiuk wrote:

 Monit is easy and is used to control states of Compute nodes. We can
 adopt it for master node.

  --
 Best regards,
 Sergii Golovatiuk,
 Skype #golserge
 IRC #holser

 On Wed, Nov 26, 2014 at 4:46 PM, Stanislaw Bogatkin 
 sbogat...@mirantis.com wrote:

 As for me - zabbix is overkill for one node. Zabbix Server + Agent +
 Frontend + DB + HTTP server, and all of it for one node? Why not use
 something that was developed for monitoring one node, doesn't have many
 deps and work out of the box? Not necessarily Monit, but something similar.

 On Wed, Nov 26, 2014 at 6:22 PM, Przemyslaw Kaminski 
 pkamin...@mirantis.com wrote:

 We want to monitor Fuel master node while Zabbix is only on slave nodes
 and not on master. The monitoring service is supposed to be installed on
 Fuel master host (not inside a Docker container) and provide basic info
 about free disk space, etc.

 P.


 On 11/26/2014 02:58 PM, Jay Pipes wrote:

 On 11/26/2014 08:18 AM, Fox, Kevin M wrote:

 So then in the end, there will be 3 monitoring systems to learn,
 configure, and debug? Monasca for cloud users, zabbix for most of the
 physical systems, and sensu or monit to be small?

 Seems very complicated.

 If not just monasca, why not the zabbix thats already being deployed?


 Yes, I had the same thoughts... why not just use zabbix since it's
 used already?

 Best,
 -jay

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 ___
 OpenStack-dev mailing 
 listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Plugins improvement

[Aleksandr Didenko]

Hi,

according to [1] you should be able to use:

puppet_modules: puppet/:/etc/puppet/modules/

This is valid string yaml parameter that should be parsed just fine.

[1]
https://github.com/stackforge/fuel-web/blob/master/tasklib/tasklib/actions/puppet.py#L61-L62

Regards
--
Alex


On Mon, Nov 24, 2014 at 12:07 PM, Dmitry Ukov du...@mirantis.com wrote:

 Hello All,
 Current implementation of plugins in Fuel unpacks plugin tarball
 into /var/www/nailgun/plugins/.
 If we implement deployment part of plugin using puppet there is a setting
 puppet_modules:

 This setting should specify path to modules folder. As soon as main
 deployment part of plugin is implemented as a Puppet module module path
 setting should be:

 puppet_modules: puppet/

 There is big probability that plugin implementation will require some
 custom resources and functions which are implemented in fuel-library (e.g.
 service config resources, stdlib functions e.t.c). So in order to use
 them plugin developer has to copy them from fuel-library into plugin (if
 i'm not missing something). This is not really convenient from my
 perspective.

 I'd like to suggest to treat puppet_modules parameter as an array and pass
 it to puppet binary as
 # puppet apply --modulepath=modulepath1:modulepath2
 This will allow to add /etc/puppet/modules as module path and use
 resources and functions form fuel-library.

 P.S.:
 puppet_modules: puppet/:/etc/puppet/moules/: - is not allowed by yaml
 parser (and yaml format I believe)

 Any suggestions here?


 --
 Kind regards
 Dmitry Ukov
 IT Engineer
 Mirantis, Inc.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] We lost some commits during upstream puppet manifests merge

[Aleksandr Didenko]

Hi,

following our docs/workflow plus writing rspec tests for every new option
we add/modify in our manifests could help with regressions. For example:

   - we add new keystone config option in openstack::keystone class -
   keystone_config {'cache/backend': value = 'keystone.cache.memcache_pool';}
   - we create new test for openstack::keystone class, something like this:
  - should
  
contain_keystone_config(cache/backend).with_value('keystone.cache.memcache_pool')

So with such test, if for some reason we lose
keystone_config(cache/backend) option, 'rake spec' would alert us about
it right away and we'll get -1 from CI. Of course we should also
implement 'rake spec' CI gate for this.

But from the other hand, if someone changes option in manifests and updates
rspec tests accordingly, then such commit will pass 'rake spec' test and we
can still lose some specific option.

 We should speed up development of some modular testing framework that
will check that corresponding change affects only particular pieces.

Such test would not catch this particular regressions with keystone_config
{'cache/backend': value = 'keystone.cache.memcache_pool';}, because even
with regression (i.e. dogpile backend) keystone was working OK. It has
passed several BVTs and custom system tests, because 'dogpile' cache
backend was working just fine while all memcached servers are up and
running. So it looks like we need some kind of tests that will ensure that
particular config options (or particular puppet resources) have some
particular values (like backend = keystone.cache.memcache_pool in [cache]
block of keystone.conf).

So I would go with rspec testing for specific resources but I would write
them in 'openstack' module. Those tests should check that needed
(nova/cinder/keystone/glance)_config resources have needed values in the
puppet catalog. Since we're not going to sync 'openstack' module with the
upstream, such tests will remain intact until we change them, and they
won't be affected by other modules sync/merge (keystone, cinder, nova, etc).

--
Regards,
Aleksandr Didenko


On Wed, Nov 19, 2014 at 5:45 PM, Vladimir Kuklin vkuk...@mirantis.com
wrote:

 Fuelers

 I am writing that we had a really sad incident - we noticed that after we
 merged upstream keystone module we lost modifications (Change-Id:
 Idfe4b54caa0d96a93e93bfff12d8b6216f83e2f1
 https://review.openstack.org/#/q/Idfe4b54caa0d96a93e93bfff12d8b6216f83e2f1,n,z)
 for memcached dogpile driver which are crucial for us. And here I can see 2
 problems:

 1) how can we ensure that we did not lose anything else?
 2) how can we ensure that this will never happen again?

 Sadly, it seems that the first question implies that we recheck all the
 upstream merge/adaptation commits by hand and check that we did not lose
 anything.

 Regarding question number 2 we do already have established process for
 upstream code merge:
 http://docs.mirantis.com/fuel-dev/develop/module_structure.html#contributing-to-existing-fuel-library-modules.
 It seems that this process had  not been established when keystone code was
 reviewed. I see two ways here:

 1) We should enforce code review workflow and specifically say that
 upstream merges can be accepted only after we have 2 '+2s' from core
 reviewers after they recheck that corresponding change does not introduce
 any regressions.
 2) We should speed up development of some modular testing framework that
 will check that corresponding change affects only particular pieces. It
 seems much easier if we split deployment into stages (oh my, I am again
 talking about granular deployment feature) and each particular commit
 affects only one of the stages, so that we can see the difference and catch
 regressions eariler.





 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 45bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Waiting for Haproxy backends

[Aleksandr Didenko]

HI,

in order to make sure some critical Haproxy backends are running (like
mysql or keystone) before proceeding with deployment, we use execs like [1]
or [2].

We're currently working on a minor improvements of those execs, but there
is another approach - we can replace those execs with puppet resource
providers and move all the iterations/loops/timeouts logic there. Also we
should fail catalog compilation/run if those resource providers are not
able to ensure needed Haproxy backends are up and running. Because there is
no point to proceed with deployment if keystone is not running, for example.

If no one objects, I can start implementing this for Fuel-6.1. We can
address it as a part of pacemaker improvements BP [3] or create a new BP.

[1]
https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/osnailyfacter/manifests/cluster_ha.pp#L551-L572
[2]
https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/openstack/manifests/ha/mysqld.pp#L28-L33
[3] https://blueprints.launchpad.net/fuel/+spec/pacemaker-improvements

Regards,
Aleksandr Didenko
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Hiera implementation

[Aleksandr Didenko]

Hi,

there is a blueprint about Hiera implementation in Fuel [1] and some
additional info plus notes [2]. So as the first step I suggest to simply
configure Hiera on Fuel master node and OS nodes before the first puppet
run and merge this into master branch. This will allow manifests developers
to start using Hiera in their modules/classes.

But we need to come up with solutions for some key moments before we can
proceed:

1) Where to ship main hiera.yaml config file?
2) What Hiera :hierarchy: should we use?
3) Where to ship hieradata yamls for fuel-library modules/classes?
4) Anything else I forgot to mention?

We should also take into account MOS versioning scheme we currently use for
puppet manifests and modules.

My suggestions:

1) We should ship hiera.yaml with our osnailyfacter module under
fuel-library project, like we do with main site.pp. So we can put
config in deployment/puppet/osnailyfacter/examples/hiera.yaml file and
symlink /etc/puppet/hiera.yaml to it [3]. This solution will support MOS
versioning, because our modules are (will be) stored in version based
directories.

2) We could use something like [4]:

   -  /etc/astute.yaml - is used as default if nothing was found
   -  /etc/puppet/hieradata/default/ - is shipped with fuel-library
   -  /etc/puppet/hieradata/override/ - allows ops team to override needed
   settings on per fact, class or module basis

In this case we'll also have to add /etc/puppet/hieradata/ into our puppet
manifests/modules versioning scheme. Maybe an easier solution would be
putting hieradata under some directory which is already versioned, for
example under /etc/fuel/6.0 ? But in this case we need to either introduce
new fact like $fuel_version and use it in Hiera :hierarchy: or setup a
symlink like /etc/fuel/current - /etc/fuel/6.0

3) Add new directory in fuel-library (for example hieradata) that will be
extracted into /etc/puppet/hieradata/default. This will allow puppet
manifest developers to add needed hiera settings along with their modules:


   - fuel-library/deployment/puppet/*my_new_module/* - puppet module
   - fuel-library/deployment/hieradata/module/*my_new_module.yaml* - hiera
   data for the module

Your input/comments are welcome and appreciated :)

[1] https://blueprints.launchpad.net/fuel/+spec/replace-parseyaml-with-hiera
[2] https://etherpad.openstack.org/p/fuel_hiera
[3] https://review.openstack.org/#/c/126559/
[4] http://pastebin.com/HH0bUtYc

Regards,
Aleksandr Didenko
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Change in fuel-library CI syntax check script

[Aleksandr Didenko]

Hi,

we've merged [1] into master, so when you create new gerrit review for
fuel-library, our CI job [2] will run syntax check (depending on file type)
for all the files found under */files/ocf/* path.

So if you want to ship OCF script as a file in your module, please put it
in MODULE/files/ocf/ directory, like in [3] and [4] for example.


[1] https://review.openstack.org/#/c/126841/5
[2] https://fuel-jenkins.mirantis.com/job/fuellib_review_syntax_check/
[3]
https://github.com/stackforge/fuel-library/tree/master/deployment/puppet/galera/files/ocf
[4]
https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/galera/manifests/init.pp#L206-L212

Regards,
Aleksandr Didenko
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Proposing Bogdan Dobrelia and Sergii Golovatiuk as core-reviewers for Fuel Library project

[Aleksandr Didenko]

+1 for both.

On Fri, Oct 10, 2014 at 5:01 PM, Igor Kalnitsky ikalnit...@mirantis.com
wrote:

 +1.

 On Fri, Oct 10, 2014 at 12:35 PM, Vladimir Kuklin vkuk...@mirantis.com
 wrote:
  Hi, Fuelers
 
  As you may have noticed our project is growing continuously. And this
  imposes a requirement of increasing amount of core reviewers. I would
 like
  to propose Bogdan Dobrelia(bogdando) and Sergii Golovatiuk(holser) as
 core
  reviewers. As you know, they have been participating actively in
  development, design and code review of the majority of project
 components as
  long as being our topmost reviewers and contributors (#2 and #3) [1 and
 2]
  (not to mention being just brilliant engineers and nice people).
 
  Please, reply to my message if you agree or disagree separately for
 Bogdan
  and Sergii (this is mandatory for existing core-reviewers).
 
  [1] http://stackalytics.com/report/contribution/fuel-library/90
  [2] http://stackalytics.com/report/contribution/fuel-library/180
 
  --
  Yours Faithfully,
  Vladimir Kuklin,
  Fuel Library Tech Lead,
  Mirantis, Inc.
  +7 (495) 640-49-04
  +7 (926) 702-39-68
  Skype kuklinvv
  45bk3, Vorontsovskaya Str.
  Moscow, Russia,
  www.mirantis.com
  www.mirantis.ru
  vkuk...@mirantis.com
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes

[Aleksandr Didenko]

Hi,

as for execution of arbitrary code across the OpenStack cluster - I was
thinking of mcollective + fact filters:

1) we need to start using mcollective facts [0] [2] - we don't
use/configure this currently
2) use mcollective execute_shell_command agent (or any other agent) with
fact filter [1]

So, for example, if we have mcollective fact called node_roles:
node_roles: compute ceph-osd

Then we can execute shell cmd on all compute nodes like this:

mco rpc execute_shell_command execute cmd=/some_script.sh -F
node_role=/compute/

Of course, we can use more complicated filters to run commands more
precisely.

[0]
https://projects.puppetlabs.com/projects/mcollective-plugins/wiki/FactsFacterYAML
[1]
https://docs.puppetlabs.com/mcollective/reference/ui/filters.html#fact-filters
[2] https://docs.puppetlabs.com/mcollective/reference/plugins/facts.html


On Wed, Sep 10, 2014 at 6:04 PM, Dmitriy Shulyak dshul...@mirantis.com
wrote:

 Hi folks,

 Some of you may know that there is ongoing work to achieve kindof
 data-driven orchestration
 for Fuel. If this is new to you, please get familiar with spec:

 https://review.openstack.org/#/c/113491/

 Knowing that running random command on nodes will be probably most usable
 type of
 orchestration extension, i want to discuss our solution for this problem.

 Plugin writer will need to do two things:

 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is
 completely configurable,
 we just need to reach agreement)

   /etc/puppet/tasks/echo/task.yaml

   with next content:

type: exec
cmd: echo 1

 2. Provide control plane with orchestration metadata

 /etc/fuel/tasks/echo_task.yaml

 controller:
  -
   task: echo
   description: Simple echo for you
   priority: 1000
 compute:
 -
   task: echo
   description: Simple echo for you
   priority: 1000

 This is done in order to separate concerns of orchestration logic and
 tasks.

 From plugin writer perspective it is far more usable to provide exact
 command in orchestration metadata itself, like:

 /etc/fuel/tasks/echo_task.yaml

 controller:
  -
   task: echo
   description: Simple echo for you
   priority: 1000
   cmd: echo 1
   type: exec

 compute:
 -
  task: echo
   description: Simple echo for you
   priority: 1000
   cmd: echo 1
   type: exec

 I would prefer to stick to the first, because there is benefits of using
 one interface between all tasks executors (puppet, exec, maybe chef), which
 will improve debuging and development process.

 So my question is first - good enough? Or second is essential type of
 plugin to support?

 If you want additional implementation details check:
 https://review.openstack.org/#/c/118311/
 https://review.openstack.org/#/c/113226/




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Using host networking for docker containers

[Aleksandr Didenko]

Hi,

we're running only 3 containers in privileged mode: cobbler, rsyslog and
mcollective. Running all the containers in privileged mode is not a good
idea for security reasons. Docker manages DNAT forwarding itself, so it
does not create any overhead for us.

 Is there any real benefits of using separate namespaces in security terms?

Of course, for example only ports specified in EXPOSE line in Dockerfile
are exposed to the host network. So if you start any additional tcp/udp
listeners inside the containers, their ports won't be accessible from the
host network.



On Sat, Aug 9, 2014 at 10:39 AM, Dmitriy Shulyak dshul...@mirantis.com
wrote:

 Hi team,

 I want to discuss benefits of using host networking [1] for docker
 containers, on master node.

 This feature was added in docker 0.11 and basicly means - reuse host
 networking stack, without
 creating separate namespace for each container.

 In my opinion it will result in much more stable install/upgrade of master
 node.

 1. There will be no need for dhcrelay/dhcrelay_monitor on host
 2. No dnat port forwarding
 3. Performance improvement for pxe boot ???

 Is there any real benefits of using separate namespaces in security terms?

 To implement this we will need:

 1. Update docker to recent version 0.12/1.x, we will do it anyway, yes?
 2. Run docker containers with --net=host

 Ofcourse it will require running containers in privileged mode, but afaik
 we are already doing this for other reasons.

 So, what do you think?

 [1] https://github.com/docker/docker/issues/2012
 [2] https://docs.docker.com/articles/networking/

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Few hot questions related to patching for openstack

[Aleksandr Didenko]

Hi,

 I think we should allow user to delete unneeded releases.

In this case user won't be able to add new nodes to the existing
environments of the same version. So we should check and warn user about
it, or simply not allow to delete releases if there are live envs with the
same version.



On Thu, Jul 3, 2014 at 3:45 PM, Dmitry Pyzhov dpyz...@mirantis.com wrote:

 So, our releases will have following versions of releases on UI:
 5.0) 2014.1
 5.0.1) 2014.1.1-5.0.1
 5.1) 2014.1.1-5.1

 And if someone install 5.0, upgrade it to 5.0.1 and then upgrade to 5.1,
 he will have three releases for each OS. I think we should allow user to
 delete unneeded releases. It also will add free space on his masternode.


 On Wed, Jul 2, 2014 at 1:34 PM, Igor Kalnitsky ikalnit...@mirantis.com
 wrote:

 Hello,

  Could you please clarify what exactly you mean by  our patches /
  our first patch?

 I mean which version should we use in 5.0.1, for example? As far as I
 understand @DmitryB, it have to be 2014.1-5.0.1. Am I right?

 Thanks,
 Igor



 On Tue, Jul 1, 2014 at 8:47 PM, Aleksandr Didenko adide...@mirantis.com
 wrote:

 Hi,

 my 2 cents:

 1) Fuel version (+1 to Dmitry)
 2) Could you please clarify what exactly you mean by our patches /
 our first patch?




 On Tue, Jul 1, 2014 at 8:04 PM, Dmitry Borodaenko 
 dborodae...@mirantis.com wrote:

 1) Puppet manifests are part of Fuel so the version of Fuel should be
 used. It is possible to have more than one version of Fuel per
 OpenStack version, but not the other way around: if we upgrade
 OpenStack version we also increase version of Fuel.

 2) Should be a combination of both: it should indicate which OpenStack
 version it is based on (2014.1.1), and version of Fuel it's included
 in (5.0.1), e.g. 2014.1.1-5.0.1. Between Fuel versions, we can have
 additional bugfix patches added to shipped OpenStack components.

 my 2c,
 -DmitryB


 On Tue, Jul 1, 2014 at 9:50 AM, Igor Kalnitsky ikalnit...@mirantis.com
 wrote:
  Hi fuelers,
 
  I'm working on Patching for OpenStack and I have the following
 questions:
 
  1/ We need to save new puppets and repos under some versioned folder:
 
  /etc/puppet/{version}/ or /var/www/nailgun/{version}/centos.
 
  So the question is which version to use? Fuel or OpenStack?
 
  2/ Which version we have to use for our patchs? We have an OpenStack
 2014.1.
  Should we use 2014.1.1 for our first patch? Or we have to use another
  format?
 
  I need a quick reply since these questions have to be solved for
 5.0.1 too.
 
  Thanks,
  Igor
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 



 --
 Dmitry Borodaenko

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Few hot questions related to patching for openstack

[Aleksandr Didenko]

Hi,

my 2 cents:

1) Fuel version (+1 to Dmitry)
2) Could you please clarify what exactly you mean by our patches / our
first patch?




On Tue, Jul 1, 2014 at 8:04 PM, Dmitry Borodaenko dborodae...@mirantis.com
wrote:

 1) Puppet manifests are part of Fuel so the version of Fuel should be
 used. It is possible to have more than one version of Fuel per
 OpenStack version, but not the other way around: if we upgrade
 OpenStack version we also increase version of Fuel.

 2) Should be a combination of both: it should indicate which OpenStack
 version it is based on (2014.1.1), and version of Fuel it's included
 in (5.0.1), e.g. 2014.1.1-5.0.1. Between Fuel versions, we can have
 additional bugfix patches added to shipped OpenStack components.

 my 2c,
 -DmitryB


 On Tue, Jul 1, 2014 at 9:50 AM, Igor Kalnitsky ikalnit...@mirantis.com
 wrote:
  Hi fuelers,
 
  I'm working on Patching for OpenStack and I have the following questions:
 
  1/ We need to save new puppets and repos under some versioned folder:
 
  /etc/puppet/{version}/ or /var/www/nailgun/{version}/centos.
 
  So the question is which version to use? Fuel or OpenStack?
 
  2/ Which version we have to use for our patchs? We have an OpenStack
 2014.1.
  Should we use 2014.1.1 for our first patch? Or we have to use another
  format?
 
  I need a quick reply since these questions have to be solved for 5.0.1
 too.
 
  Thanks,
  Igor
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 



 --
 Dmitry Borodaenko

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Removing old node logs

[Aleksandr Didenko]

Hi,

If user runs some experiments with creating/deleting clusters, then taking
care of old logs is under user's responsibility, I suppose. Fuel configures
log rotation with compression for remote logs, so old logs will be gzipped
and will not take much space.

In case of additional boolean parameter, the default value should be
0-don't touch old logs.

--
Regards,
Alex


On Tue, Jun 24, 2014 at 4:07 PM, Vladimir Kozhukalov 
vkozhuka...@mirantis.com wrote:

 Guys,

 What do you think of removing node logs on master node right after
 removing node from cluster?

 The issue is when user do experiments he creates and deletes clusters and
 old unused directories remain and take disk space. On the other hand, it is
 not so hard to imaging the situation when user would like to be able to
 take a look in old logs.

 My suggestion here is to add a boolean parameter into settings which will
 manage this piece of logic (1-remove old logs, 0-don't touch old logs).

 Thanks for your opinions.

 Vladimir Kozhukalov

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Removing old node logs

[Aleksandr Didenko]

Yeah, I thought about diagnostic snapshot too. Maybe it would be better to
implement per-environment diagnostic snapshots? I.e. add diagnostic
snapshot generate/download buttons/links in the environment actions tab.
Such snapshot would contain info/logs about Fuel master node and nodes
assigned to the environment only.


On Tue, Jun 24, 2014 at 6:27 PM, Igor Kalnitsky ikalnit...@mirantis.com
wrote:

 Hi guys,

 What about our diagnostic snapshot?

 I mean we're going to make snapshot of entire /var/log and obviously
 this old logs will be included in snapshot. Should we skip theem or
 such situation is ok?

 - Igor




 On Tue, Jun 24, 2014 at 5:57 PM, Aleksandr Didenko adide...@mirantis.com
 wrote:

 Hi,

 If user runs some experiments with creating/deleting clusters, then
 taking care of old logs is under user's responsibility, I suppose. Fuel
 configures log rotation with compression for remote logs, so old logs will
 be gzipped and will not take much space.

 In case of additional boolean parameter, the default value should be
 0-don't touch old logs.

 --
 Regards,
 Alex


 On Tue, Jun 24, 2014 at 4:07 PM, Vladimir Kozhukalov 
 vkozhuka...@mirantis.com wrote:

 Guys,

 What do you think of removing node logs on master node right after
 removing node from cluster?

 The issue is when user do experiments he creates and deletes clusters
 and old unused directories remain and take disk space. On the other hand,
 it is not so hard to imaging the situation when user would like to be able
 to take a look in old logs.

 My suggestion here is to add a boolean parameter into settings which
 will manage this piece of logic (1-remove old logs, 0-don't touch old logs).

 Thanks for your opinions.

 Vladimir Kozhukalov

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Removing old node logs

[Aleksandr Didenko]

Yes, of course, snapshot for all nodes at once (like currently) should also
be available.


On Tue, Jun 24, 2014 at 7:27 PM, Igor Kalnitsky ikalnit...@mirantis.com
wrote:

 Hello,

 @Aleks, it's a good idea to make snapshot per environment, but I think
 we can keep functionality to make snapshot for all nodes at once too.

 - Igor


 On Tue, Jun 24, 2014 at 6:38 PM, Aleksandr Didenko adide...@mirantis.com
 wrote:

 Yeah, I thought about diagnostic snapshot too. Maybe it would be better
 to implement per-environment diagnostic snapshots? I.e. add diagnostic
 snapshot generate/download buttons/links in the environment actions tab.
 Such snapshot would contain info/logs about Fuel master node and nodes
 assigned to the environment only.


 On Tue, Jun 24, 2014 at 6:27 PM, Igor Kalnitsky ikalnit...@mirantis.com
 wrote:

 Hi guys,

 What about our diagnostic snapshot?

 I mean we're going to make snapshot of entire /var/log and obviously
 this old logs will be included in snapshot. Should we skip theem or
 such situation is ok?

 - Igor




 On Tue, Jun 24, 2014 at 5:57 PM, Aleksandr Didenko 
 adide...@mirantis.com wrote:

 Hi,

 If user runs some experiments with creating/deleting clusters, then
 taking care of old logs is under user's responsibility, I suppose. Fuel
 configures log rotation with compression for remote logs, so old logs will
 be gzipped and will not take much space.

 In case of additional boolean parameter, the default value should be
 0-don't touch old logs.

 --
 Regards,
 Alex


 On Tue, Jun 24, 2014 at 4:07 PM, Vladimir Kozhukalov 
 vkozhuka...@mirantis.com wrote:

 Guys,

 What do you think of removing node logs on master node right after
 removing node from cluster?

 The issue is when user do experiments he creates and deletes clusters
 and old unused directories remain and take disk space. On the other hand,
 it is not so hard to imaging the situation when user would like to be able
 to take a look in old logs.

 My suggestion here is to add a boolean parameter into settings which
 will manage this piece of logic (1-remove old logs, 0-don't touch old 
 logs).

 Thanks for your opinions.

 Vladimir Kozhukalov

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev