from:"Evgeniy L"

Because it doesn't make much sense for a plugin developer to have scripts
to build packages for installation on slave nodes. For default template
it's better to have something minimalistic and the rest of the tasks
commented, otherwise it may create a lot of confusion.

On Wed, Mar 9, 2016 at 6:37 PM, Ilya Kutukov <ikutu...@mirantis.com> wrote:

> Talking about templates I mean plugins generated by FBP from the
> `templates` folder using command you mentioned above.
>
> Why not to extend (not replace) template-generated plugins with additional
> data to provide right coverage?
>
> On Wed, Mar 9, 2016 at 6:23 PM, Evgeniy L <e...@mirantis.com> wrote:
>
>> Ilya,
>>
>> What do you mean by "templates" the plugin which is create by just "fpb
>> --create plugin-name"?
>> It doesn't cover enough, package installation and all range of tasks
>> executions.
>>
>> Thanks,
>>
>> On Wed, Mar 9, 2016 at 6:16 PM, Ilya Kutukov <ikutu...@mirantis.com>
>> wrote:
>>
>>> Igor, i completely agree, actually plugin examples is almost a
>>> copy-paste.
>>>
>>> The only thing i see useful in the built plugins example is the ability
>>> to point some code lines, discussing plugin design and writing
>>> documentation. Why not to generate this examples automatically from
>>> templates?
>>>
>>> Also, as developer and administrator i love to use examples/templates
>>> where all essential settings and options are persist but commented (e.g.
>>> ProFTPD or Apache) and i could uncomment and copy-paste them not being
>>> afraid of typos. Also it allows to keep documentation actual and head
>>> documentation small. Duplication of inline documentation between examples
>>> and templates making things more weird and overshadows idea of inline
>>> documentation itself.
>>>
>>> Eugeny, why not to run integration tests against plugins generated from
>>> templates? It's will be even better integration tests.
>>>
>>>
>>>
>>> On Mon, Mar 7, 2016 at 4:49 PM, Igor Kalnitsky <ikalnit...@mirantis.com>
>>> wrote:
>>>
>>>> > and really lowering barriers for people who just begin create plugins.
>>>>
>>>> Nonsense. First, people usually create them via running `fpb --create
>>>> plugin-name` that generates plugin boilerplate. And that boilerplate
>>>> won't contain that changes.
>>>>
>>>> Second, if people ain't smart enough to change few lines in
>>>> `metadata.yaml` of generated boilerplate to make it work with latest
>>>> Fuel, maybe it's better for them to do not develop plugins at all?
>>>>
>>>> On Fri, Mar 4, 2016 at 2:24 PM, Stanislaw Bogatkin
>>>> <sbogat...@mirantis.com> wrote:
>>>> > +1 to maintain example plugins. It is easy enough and really lowering
>>>> > barriers for people who just begin create plugins.
>>>> >
>>>> > On Fri, Mar 4, 2016 at 2:08 PM, Matthew Mosesohn <
>>>> mmoses...@mirantis.com>
>>>> > wrote:
>>>> >>
>>>> >> Igor,
>>>> >>
>>>> >> It seems you are proposing an IKEA approach to plugins. Take Fuel's
>>>> >> example plugin, add in the current Fuel release, and then build it.
>>>> We
>>>> >> maintained these plugins in the past, but now it should a manual step
>>>> >> to test it out on the current release.
>>>> >>
>>>> >> What would be a more ideal situation that meets the needs of users
>>>> and
>>>> >> QA? Right now we have failed tests until we can decide on a solution
>>>> >> that works for everybody.
>>>> >>
>>>> >> On Fri, Mar 4, 2016 at 1:26 PM, Igor Kalnitsky <
>>>> ikalnit...@mirantis.com>
>>>> >> wrote:
>>>> >> > No, this is a wrong road to go.
>>>> >> >
>>>> >> > What if in Fuel 10 we drop v1 plugins support? What should we do?
>>>> >> > Remove v1 example from source tree? That doesn't seem good to me.
>>>> >> >
>>>> >> > Example plugins are only examples. The list of supported releases
>>>> must
>>>> >> > be maintained on system test side, and system tests must inject
>>>> that
>>>> >> > information into plugin's metadata.yaml and test it.
>&

Re: [openstack-dev] [fuel][plugins] Should we maintain example plugins?

Ilya,

What do you mean by "templates" the plugin which is create by just "fpb
--create plugin-name"?
It doesn't cover enough, package installation and all range of tasks
executions.

Thanks,

On Wed, Mar 9, 2016 at 6:16 PM, Ilya Kutukov  wrote:

> Igor, i completely agree, actually plugin examples is almost a copy-paste.
>
> The only thing i see useful in the built plugins example is the ability to
> point some code lines, discussing plugin design and writing documentation.
> Why not to generate this examples automatically from templates?
>
> Also, as developer and administrator i love to use examples/templates
> where all essential settings and options are persist but commented (e.g.
> ProFTPD or Apache) and i could uncomment and copy-paste them not being
> afraid of typos. Also it allows to keep documentation actual and head
> documentation small. Duplication of inline documentation between examples
> and templates making things more weird and overshadows idea of inline
> documentation itself.
>
> Eugeny, why not to run integration tests against plugins generated from
> templates? It's will be even better integration tests.
>
>
>
> On Mon, Mar 7, 2016 at 4:49 PM, Igor Kalnitsky 
> wrote:
>
>> > and really lowering barriers for people who just begin create plugins.
>>
>> Nonsense. First, people usually create them via running `fpb --create
>> plugin-name` that generates plugin boilerplate. And that boilerplate
>> won't contain that changes.
>>
>> Second, if people ain't smart enough to change few lines in
>> `metadata.yaml` of generated boilerplate to make it work with latest
>> Fuel, maybe it's better for them to do not develop plugins at all?
>>
>> On Fri, Mar 4, 2016 at 2:24 PM, Stanislaw Bogatkin
>>  wrote:
>> > +1 to maintain example plugins. It is easy enough and really lowering
>> > barriers for people who just begin create plugins.
>> >
>> > On Fri, Mar 4, 2016 at 2:08 PM, Matthew Mosesohn <
>> mmoses...@mirantis.com>
>> > wrote:
>> >>
>> >> Igor,
>> >>
>> >> It seems you are proposing an IKEA approach to plugins. Take Fuel's
>> >> example plugin, add in the current Fuel release, and then build it. We
>> >> maintained these plugins in the past, but now it should a manual step
>> >> to test it out on the current release.
>> >>
>> >> What would be a more ideal situation that meets the needs of users and
>> >> QA? Right now we have failed tests until we can decide on a solution
>> >> that works for everybody.
>> >>
>> >> On Fri, Mar 4, 2016 at 1:26 PM, Igor Kalnitsky <
>> ikalnit...@mirantis.com>
>> >> wrote:
>> >> > No, this is a wrong road to go.
>> >> >
>> >> > What if in Fuel 10 we drop v1 plugins support? What should we do?
>> >> > Remove v1 example from source tree? That doesn't seem good to me.
>> >> >
>> >> > Example plugins are only examples. The list of supported releases
>> must
>> >> > be maintained on system test side, and system tests must inject that
>> >> > information into plugin's metadata.yaml and test it.
>> >> >
>> >> > Again, I don't say we shouldn't test plugins. I say, tests should be
>> >> > responsible for preparing plugins. I can say even more: tests should
>> >> > not rely on what is produced by plugins, since it's something that
>> >> > could be changed and tests start failing.
>> >> >
>> >> > On Thu, Mar 3, 2016 at 7:54 PM, Swann Croiset > >
>> >> > wrote:
>> >> >> IMHO it is important to keep plugin examples and keep testing them,
>> >> >> very
>> >> >> valuable for plugin developers.
>> >> >>
>> >> >> For example, I've encountered [0] the case where "plugin as role"
>> >> >> feature
>> >> >> wasn't easily testable with fuel-qa because not compliant with the
>> last
>> >> >> plugin data structure,
>> >> >> and more recently we've spotted a regression [1] with
>> "vip-reservation"
>> >> >> feature introduced by a change in nailgun.
>> >> >> These kind of issues are time consuming for plugin developers and
>> >> >> can/must
>> >> >> be avoided by testing them.
>> >> >>
>> >> >> I don't even understand why the question is raised while fuel
>> plugins
>> >> >> are
>> >> >> supposed to be supported and more and more used [3], even by murano
>> [4]
>> >> >> ...
>> >> >>
>> >> >> [0] https://bugs.launchpad.net/fuel/+bug/1543962
>> >> >> [1] https://bugs.launchpad.net/fuel/+bug/1551320
>> >> >> [3]
>> >> >>
>> >> >>
>> http://lists.openstack.org/pipermail/openstack-dev/2016-February/085636.html
>> >> >> [4] https://review.openstack.org/#/c/286310/
>> >> >>
>> >> >> On Thu, Mar 3, 2016 at 3:19 PM, Matthew Mosesohn
>> >> >> 
>> >> >> wrote:
>> >> >>>
>> >> >>> Hi Fuelers,
>> >> >>>
>> >> >>> I would like to bring your attention a dilemma we have here. It
>> seems
>> >> >>> that there is a dispute as to whether we should maintain the
>> releases
>> >> >>> list for example plugins[0]. In this case, this is for adding
>> version
>> >> >>> 9.0 to the list.
>> >>

Re: [openstack-dev] [fuel][plugins] Should we maintain example plugins?

Hi,

Plugin examples mustn't be removed, those plugins are part of integration
tests for fuel plugin builder, which should be able to build any version of
plugin.

So there are two ways to solve the problem:
1. Before test run update compatibility matrix for plugins automatically.
2. Continue fixing plugin examples.

We may try to do 2nd automatically (proposing a patch with fixed version)
on version change in openstack.yaml.

Thanks,

On Tue, Mar 8, 2016 at 3:50 PM, Anastasia Urlapova 
wrote:

> Igor,
> I agree with folks, that we have to fix plugin. You didn't mention any
> real reasons, why we should change workflow with example-plugin right now
> in the middle of release. If we are keeping support of the example plugin
> in 9.0 it should work from box w/o any black magic in other places.
>
>
> Nastya.
>
> On Mon, Mar 7, 2016 at 6:33 PM, Simon Pasquier 
> wrote:
>
>> Yet another example [1] of why a dummy/example plugin should be
>> integrated in the Fuel CI process: the current version of Fuel is broken
>> for (almost) all plugins since a week at least and no one noticed.
>> Regards,
>> Simon
>>
>> [1] https://bugs.launchpad.net/fuel/+bug/1554095
>>
>> On Mon, Mar 7, 2016 at 3:16 PM, Simon Pasquier 
>> wrote:
>>
>>> What about maintaining a dummy plugin (eg running only one or two very
>>> simple tasks) as a standalone project for the purpose of QA?
>>> IMO it would make more sense than having those example plugins in the
>>> fuel-plugins project...
>>> Regards,
>>> Simon
>>>
>>> On Mon, Mar 7, 2016 at 2:49 PM, Igor Kalnitsky 
>>> wrote:
>>>
 > and really lowering barriers for people who just begin create plugins.

 Nonsense. First, people usually create them via running `fpb --create
 plugin-name` that generates plugin boilerplate. And that boilerplate
 won't contain that changes.

 Second, if people ain't smart enough to change few lines in
 `metadata.yaml` of generated boilerplate to make it work with latest
 Fuel, maybe it's better for them to do not develop plugins at all?

 On Fri, Mar 4, 2016 at 2:24 PM, Stanislaw Bogatkin
  wrote:
 > +1 to maintain example plugins. It is easy enough and really lowering
 > barriers for people who just begin create plugins.
 >
 > On Fri, Mar 4, 2016 at 2:08 PM, Matthew Mosesohn <
 mmoses...@mirantis.com>
 > wrote:
 >>
 >> Igor,
 >>
 >> It seems you are proposing an IKEA approach to plugins. Take Fuel's
 >> example plugin, add in the current Fuel release, and then build it.
 We
 >> maintained these plugins in the past, but now it should a manual step
 >> to test it out on the current release.
 >>
 >> What would be a more ideal situation that meets the needs of users
 and
 >> QA? Right now we have failed tests until we can decide on a solution
 >> that works for everybody.
 >>
 >> On Fri, Mar 4, 2016 at 1:26 PM, Igor Kalnitsky <
 ikalnit...@mirantis.com>
 >> wrote:
 >> > No, this is a wrong road to go.
 >> >
 >> > What if in Fuel 10 we drop v1 plugins support? What should we do?
 >> > Remove v1 example from source tree? That doesn't seem good to me.
 >> >
 >> > Example plugins are only examples. The list of supported releases
 must
 >> > be maintained on system test side, and system tests must inject
 that
 >> > information into plugin's metadata.yaml and test it.
 >> >
 >> > Again, I don't say we shouldn't test plugins. I say, tests should
 be
 >> > responsible for preparing plugins. I can say even more: tests
 should
 >> > not rely on what is produced by plugins, since it's something that
 >> > could be changed and tests start failing.
 >> >
 >> > On Thu, Mar 3, 2016 at 7:54 PM, Swann Croiset <
 scroi...@mirantis.com>
 >> > wrote:
 >> >> IMHO it is important to keep plugin examples and keep testing
 them,
 >> >> very
 >> >> valuable for plugin developers.
 >> >>
 >> >> For example, I've encountered [0] the case where "plugin as role"
 >> >> feature
 >> >> wasn't easily testable with fuel-qa because not compliant with
 the last
 >> >> plugin data structure,
 >> >> and more recently we've spotted a regression [1] with
 "vip-reservation"
 >> >> feature introduced by a change in nailgun.
 >> >> These kind of issues are time consuming for plugin developers and
 >> >> can/must
 >> >> be avoided by testing them.
 >> >>
 >> >> I don't even understand why the question is raised while fuel
 plugins
 >> >> are
 >> >> supposed to be supported and more and more used [3], even by
 murano [4]
 >> >> ...
 >> >>
 >> >> [0] https://bugs.launchpad.net/fuel/+bug/1543962
 >> >> [1] https://bugs.launchpad.net/fuel/+bug/1551320
 >> >> [3]

Re: [openstack-dev] [Fuel] UI code freeze

Hi,

Thank you for your work, really happy to see it done. So as far as I can
see from now on in fuel-web repository we have only Nailgun project. Is it
correct?

On Fri, Feb 26, 2016 at 3:53 PM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> Dear colleagues,
>
> We are ready for moving UI javascript code to a separate git repository.
> Since now all review requests for fuel-web@master that are related to UI
> must be marked -2 and later abandoned. Once new project fuel-ui is created
> those changes should be backported to this new repository.
>
> Checklist:
>
>
>- Launchpad UI bug: https://bugs.launchpad.net/fuel/+bug/1471763
>- openstack-infra
>   - project-config https://review.openstack.org/#/c/260455 (ON REVIEW)
>   - governance https://review.openstack.org/#/c/285270/ (ON REVIEW)
>- fuel-ci job https://review.fuel-infra.org/#/c/15487 (ON REVIEW)
>- label jenkins slaves for the new job (ci team)
>- UI directory freeze (DONE)
>- prepare upstream https://github.com/kozhukalov/fuel-ui.git (DONE)
>- waiting for openstack-infra patches to be merged
>- .gitreview .gitignore MAINTAINERS (TODO)
>- rpm spec (TODO)
>- custom ci jobs (TODO)
>- fuel-main (TODO)
>- packaging ci (TODO)
>- remove old files (TODO)
>
>
> Vladimir Kozhukalov
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Solar] Integration in 9.0

2016-02-26 Thread Evgeniy L

Hi Jedrzej,

>> Maybe instead blueprint in 1st step should we create full blown
fuel-spec?

If there is anything to be discussed in implementation, or there are
different options to do it, it's better to have blueprint and spec, so
everybody will be able to see what the integration looks like.

Thanks,

On Fri, Feb 26, 2016 at 4:40 PM, Jedrzej Nowak  wrote:

> As you may now we should integrate solar with fuel in 9.0. Integration in
> 9.0 will be very experimental and it will not affect fuel except: spec
> https://review.openstack.org/283600
>  and https://review.openstack.org/#/c/284293/
>
> We will introduce 2 new packages solar and fuel2solar ( + some
> dependencies) in Centos7, they will be available in mos-master
> repository. We would like to include not installed solar and fuel2solar in
> 9.0 ISO.
>
> Predicted UX:
> - user will need to install 2 rpm packages: solar and fuel2solar
> - user will configure everything in fuel-web and then switch to solar CLI
> before clicking "deploy" button
>
> Our current plan is:
> 1. create blueprint about introducing these 2 additional packages (and
> requirements)
> 2. then should we describe everything again in openstack-dev ML
> 3. describe this everything in fuel docs
>
> Maybe instead blueprint  in 1st step should we create full blown fuel-spec
> ? That spec obviously will not require any QA activities.
>
>
> --
> Warm regards,
> Jędrzej Nowak
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Question about Feature Freeze Exceptions for non-invasive features

2016-02-26 Thread Evgeniy L

Hi Dmitry,

The thing is it may not require changes in Fuel for 10.0 too, ideally we
would like to add a couple of new gates to OpenStack infra, so they will be
enabled for external extensions when there are any.

Thanks,

On Thu, Feb 25, 2016 at 8:55 PM, Dmitry Borodaenko <dborodae...@mirantis.com
> wrote:

> If the spec is not going to require code changes in 9.0/Mitaka, why not
> simply target it for 10.0/Newton?
>
> On Thu, Feb 25, 2016 at 05:21:55PM +0300, Evgeniy L wrote:
> > Hi,
> >
> > We have a spec where we are trying to do research and describe how we are
> > going to test extensions [0], we will not be able to land it before
> feature
> > freeze, so should we get feature freeze exception for it? Or it will not
> be
> > a problem to merge it even after FF?
> >
> > The spec itself is more about the process which we are going to follow
> > during extensions development, so there should be no changes for current
> > projects.
> >
> > Thanks,
> >
> > [0] https://review.openstack.org/#/c/281749/
>
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Question about Feature Freeze Exceptions for non-invasive features

2016-02-25 Thread Evgeniy L

Hi,

We have a spec where we are trying to do research and describe how we are
going to test extensions [0], we will not be able to land it before feature
freeze, so should we get feature freeze exception for it? Or it will not be
a problem to merge it even after FF?

The spec itself is more about the process which we are going to follow
during extensions development, so there should be no changes for current
projects.

Thanks,

[0] https://review.openstack.org/#/c/281749/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Bareon] Weekly update

2016-02-22 Thread Evgeniy L

Hi,

Here is weekly update from Bareon team.

1. 3 specs from Cray team were merged [0], roadmap was properly adjusted [1]
2. Changing deployment data using extensions in Nailgun [2], spec is
merged, code is on review [3]
3. Extensions testing procedure, spec is in progress [4]
4. Dynamic allocator [5] is in progress, finished writing base integration
tests, started refactoring to make code production ready
5. Pluggable actions, tests are in progress [6]

Also provided a help with bug triaging [7] and consultation for a couple of
Fuel blueprints [8], [9].

Thanks,

[0] https://review.openstack.org/#/q/project:openstack/bareon-specs
[1] https://wiki.openstack.org/wiki/Bareon/Roadmap
[2] https://review.openstack.org/#/c/274653/
[3] https://review.openstack.org/#/c/272977/
[4] https://review.openstack.org/#/c/281749/
[5] https://blueprints.launchpad.net/bareon/+spec/dynamic-allocation
[6] https://blueprints.launchpad.net/bareon/+spec/pluggable-do-actions
[7] https://bugs.launchpad.net/fuel/+bug/1546604
[8] https://blueprints.launchpad.net/fuel/+spec/multipath-disks-support
[9] https://blueprints.launchpad.net/fuel/+spec/fuel-nonroot-openstack-nodes
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Wildcards instead of

2016-02-20 Thread Evgeniy L

Hi,

+1 to Igor, plugin developer should be able to granularly define what
she/he wants to be executed on the node, without assumptions from our side.

`exclude` - field doesn't look like a good solution to me, it will be hard
to support and migrate plugins to newer version OpenStack release.

I would suggest to solve it differently, lets define namespaces, which we
will be able to use to identify tasks from core, with prefix
"std:role-name" or "core:role-name", also in the same way plugin or a group
of plugins will be able to define their namespaces "lma:role-name" or
"detached:keystone", so for library tasks you will have something like that:

groups: ['/std:.*/']

or

groups: ['/core:.*/']

It's natural to have such thing, because it's similar to what is in
Python/Ruby (modules) or in C++ (namespaces).

Thanks,

On Fri, Feb 19, 2016 at 6:02 PM, Aleksandr Didenko 
wrote:

> > I vote to abandon it. Let's do not break existing plugins, and do not
> > add *undo* tasks for plugin developers. If they want to configure
> > network, they'll ask it explicitly.
>
> +1 to this gentleman. It's safe to add wildcards only to tasks that were
> moved from pre/post deployment stages, which were executed everywhere
> anyway.
>
> Regards,
> Alex
>
> On Fri, Feb 19, 2016 at 3:22 PM, Bulat Gaifullin 
> wrote:
>
>>
>> > On 19 Feb 2016, at 17:09, Igor Kalnitsky 
>> wrote:
>> >
>> > Kyrylo G. wrote:
>> >> So who is voting for the path to be abandoned?
>> >
>> > I vote to abandon it. Let's do not break existing plugins, and do not
>> > add *undo* tasks for plugin developers. If they want to configure
>> > network, they'll ask it explicitly.
>> >
>> >
>> > Kyrylo G. wrote:
>> >> By the way, there is already a task running by the wildcard:
>> >>
>> https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/fuel_pkgs/tasks.yaml#L4
>> >
>> > Yes, exactly, but the thing is that our original task for setuping
>> > repos was executed on all nodes before, including ones provided by
>> > plugins. Making it executing on core nodes only may break plugins that
>> > rely on it. So generally, it's about backward compatibility.
>> >
>> >
>> > Bulat G. wrote:
>> >> This tasks should run on all nodes and it does not matter, the node
>> >> has role from plugin or core-role.
>> >
>> > Nope, they shouldn't. Why do I need to install the following packages
>> >
>> >  'screen',
>> >  'tmux',
>> >  'htop',
>> >  'tcpdump',
>> >  'strace',
>> >  'fuel-misc',
>> >  'man-db',
>> >  'fuel-misc',
>> >  'fuel-ha’
>> >
>> It is big problem?
>>
>> > if I have no plans to use them? As a deployer engineer, I'd prefer to
>> > keep my role as clear as possible, and decide what to install in my
>> > own way.
>>
>> IMO: The plugin developer wants to install additional applications to
>> extend functionality, It do not want configure low-level things, like
>> specify some banch of task for configure network, configure repositories
>> etc.
>> How can we manage new node if network is not configured or fuel-agent is
>> not installed?
>>
>> >
>> >
>> > On Fri, Feb 19, 2016 at 1:06 PM, Bulat Gaifullin
>> >  wrote:
>> >> +1 to use wildcards for common tasks like netconfig and setup
>> repositories.
>> >> This tasks should run on all nodes and it does not matter, the node
>> has role
>> >> from plugin or core-role.
>> >> In my opinion we should one approach for basic configuration of node.
>> >>
>> >> Regards,
>> >> Bulat Gaifullin
>> >> Mirantis Inc.
>> >>
>> >>
>> >>
>> >> On 19 Feb 2016, at 13:36, Kyrylo Galanov 
>> wrote:
>> >>
>> >> Hi,
>> >>
>> >> So who is voting for the path to be abandoned?
>> >>
>> >> By the way, there is already a task running by the wildcard:
>> >>
>> https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/fuel_pkgs/tasks.yaml#L4
>> >> However, it this case it might work with plugins.
>> >>
>> >> Best regards,
>> >> Kyrylo
>> >>
>> >> On Fri, Feb 19, 2016 at 1:09 AM, Igor Kalnitsky <
>> ikalnit...@mirantis.com>
>> >> wrote:
>> >>>
>> >>> Hey Kyrylo,
>> >>>
>> >>> As it was mentioned in the review: you're about to break roles defined
>> >>> by plugins. That's not good move, I believe.
>> >>>
>> >>> Regarding 'exclude' directive, I have no idea what you're talking
>> >>> about. We don't support it now, and, anyway, there should be no
>> >>> difference between roles defined by plugins and core roles.
>> >>>
>> >>> - Igor
>> >>>
>> >>> On Thu, Feb 18, 2016 at 12:53 PM, Kyrylo Galanov <
>> kgala...@mirantis.com>
>> >>> wrote:
>>  Hello,
>> 
>>  We are about to switch to wildcards instead of listing all groups in
>>  tasks
>>  explicitly [0].
>>  This change must make deployment process more obvious for developers.
>>  However, it might lead to confusion when new groups are added either
>> by
>>  plugin or fuel team in

Re: [openstack-dev] [fuel][nailgun][volume-manager][fuel-agent] lvm metadata size value. why was it set to 64M?

2016-02-18 Thread Evgeniy L

Hi Alexander,

I was trying to trace the change and found 3 year old commit, yes it's hard
to recover the reason [0].
So what we should ask is what is a right way to calculate lvm metadata size
and change this behaviour.

I would suggest at least explicitly set metadata size on Nailgun side to
the same amount we have in the agent (until better size is found). Plus
explicitly reserve some amount of space based on io-optimal of specific
disk.

Thanks,

[0]
https://github.com/Mirantis/fuelweb/commit/d4d14b528b76b8e9fcbca51d3047a3884792d69f
[1] https://www.redhat.com/archives/linux-lvm/2012-April/msg00024.html

On Wed, Feb 17, 2016 at 8:51 PM, Alexander Gordeev 
wrote:

> Hi,
>
> Apparently, nailgun assumes that lvm metadata size is always set to 64M [1]
>
> It seems that it was defined here since the early beginning of nailgun as
> a project, therefore it's impossible to figure out for what purposes that
> was done as early commit messages are not so informative.
>
> According to the documentation (man lvm.conf):
>
>   pvmetadatasize — Approximate number of sectors to set aside
> for each copy of the metadata. Volume groups with large numbers  of
> physical  or  logical  volumes,  or  volumes groups containing complex
> logical volume structures will need additional space for their metadata.
> The metadata areas are treated as circular buffers, so unused space becomes
> filled with an archive of the most recent previous versions of the metadata.
>
>
> The default value is set to 255 sectors. (128KiB)
>
> Quotation from particular lvm.conf sample:
> # Approximate default size of on-disk metadata areas in sectors.
> # You should increase this if you have large volume groups or
> # you want to retain a large on-disk history of your metadata changes.
>
> # pvmetadatasize = 255
>
>
> nailgun's volume manager calculates sizes of logical volumes within one
> physical volume group and takes into account the size of lvm metadata [2].
>
> However, due to logical volumes size gets rounded to the nearest multiple
> of PE size (which is 4M usually), fuel-agent always ends up with the lack
> of free space when creating logical volumes exactly in accordance with
> partitioning scheme is generated by volume manager.
> Thus, tricky logic was added into fuel-agent [3] to bypass that flaw.
> Since 64M is way too big value when compared with typical one, fuel-agent
> silently reduces the size of lvm metadata by 8M and then partitioning
> always goes smooth.
>
> Consequently, almost each physical volume group remains only 4M of free
> space. It worked fine on old good HDDs.
>
> But when the time comes to use any FC/HBA/HW RAID block storage device
> which is occasionally reporting relatively huge values for minimal io size
> and optimal io size exposed in sysfs, then fuel-agent might end up with the
> lack of free space once again due to logical volume alignments within
> physical volume group [4]. Those alignments have been done by LVM
> automatically with respect to those values [5]
>
> As I'm going to trade off some portion of reserved amount of disk space
> for storing lvm metadata for the sake of logical volume alignments, here're
> the questions:
>
> * why was lvm metadata set to 64M?
> * could someone shed more light on any obvious reasons/needs hidden behind
> that?
> * what is the minimal size of lvm metadata we'll be happy with?
> * the same question for the optimal size.
>
>
> [1]
> https://github.com/openstack/fuel-web/blob/6bd08607c6064e99ad2ed277b1c17d7b23b13c8a/nailgun/nailgun/extensions/volume_manager/manager.py#L824
> [2]
> https://github.com/openstack/fuel-web/blob/6bd08607c6064e99ad2ed277b1c17d7b23b13c8a/nailgun/nailgun/extensions/volume_manager/manager.py#L867-L875
> [3]
> https://github.com/openstack/fuel-agent/commit/c473202d4db774b0075b8d9c25f217068f7c1727
> [4] https://bugs.launchpad.net/fuel/+bug/1546049
> [5] http://people.redhat.com/msnitzer/docs/io-limits.txt
>
>
> Thanks,
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-16 Thread Evgeniy L

Hi,

I have some comments on CI for plugins, currently there is a good
instruction on how to install your own CI and using fuel-qa write your own
tests [0], since just running BVT is not enough to make sure that plugin
works, we should provide a way for a plugin developer to easily extend
checks/asserts which are done during BVT by just putting python files into
plugin-name/tests directory. This way will be able to install plugin
(enable it) and perform plugin specific checks.

Thanks,

[0] https://wiki.openstack.org/wiki/Fuel/Plugins#CI

On Wed, Feb 3, 2016 at 5:27 AM, Dmitry Borodaenko 
wrote:

> It has been over a year since pluggable architecture was introduced in
> Fuel 6.0, and I think it's safe to declare it an unmitigated success. A
> search for "fuel-plugin" on GitHub brings up 167 repositories [0],
> there's 63 Fuel plugin repositories on review.openstack.org [1], 25 Fuel
> plugins are listed in the DriverLog [2].
>
> [0] https://github.com/search?q=fuel-plugin-
> [1]
> https://review.openstack.org/#/admin/projects/?filter=openstack%252Ffuel-plugin-
> [2] http://stackalytics.com/report/driverlog?project_id=openstack%2Ffuel
>
> Even though the plugin engine is not yet complete (there still are
> things you can do in Fuel core that you cannot do in a plugin), dozens
> of deployers and developers [3] used it to expand Fuel capabilities
> beyond the limitations of our default reference architecture.
>
> [3] http://stackalytics.com/report/contribution/fuel-plugins-group/360
>
> There's a noticeable bump in contributions around October 2015 after
> Fuel 7.0 was released, most likely inspired by the plugin engine
> improvements introduced in that version [4]. As we continue to expand
> plugins capabilities, I expect more and more plugins to appear.
>
> [4]
> https://git.openstack.org/cgit/openstack/fuel-docs/tree/pages/release-notes/v7-0/new_features/plugins.rst?h=stable/7.0
>
> The question of how useful exactly all those plugins are is a bit harder
> to answer. DriverLog isn't much help: less than half of Fuel plugins
> hosted on OpenStack infrastructure are even registered there, and of
> those that are, only 6 have CI jobs with recent successful runs. Does
> this mean that 90% of Fuel plugins are broken and unmaintained? Not
> necessarily, but it does mean that we have no way to tell.
>
> An even harder question is: once we determine that some plugins are more
> equal than others, what should we do about the less useful and the less
> actively maintained?
>
> To objectively answer both questions, we need to define support levels
> for Fuel plugins and set some reasonable expectations about how plugins
> can qualify for each level.
>
> Level 3. Plugin is not actively supported
>
> I believe that having hundreds of Fuel plugins out there on GitHub and
> elsewhere is great, and we should encourage people to create more of
> those and do whatever they like with them. Even a single-commit "deploy
> and forget" plugin is useful as an idea, a source of inspiration, and a
> starting point for other people who might want to take it further.
>
> At this level, there should be zero expectations and zero obligations
> between Fuel plugin writers and OpenStack community. At the moment, Fuel
> plugins developers guide recommends [5] to request a Gerrit repo in the
> openstack/ namespace and set up branches, tags, CI, and a code review
> process around it, aligned with OpenStack development process. Which is
> generally a good idea, except for all the cases where it's too much
> overhead and ends up not being followed closely enough to be useful.
>
> [5] https://wiki.openstack.org/wiki/Fuel/Plugins#Repo
>
> Instead of vague blanket recommendations, we should explictly state that
> it's fine to do none of that and just stay on GitHub, and that if you
> intend to move to the next level and actively maintain your plugin, and
> expect support with that from Fuel developers and other OpenStack
> projects, these recommendations are not optional and must be fulfilled.
>
> Level 2. Plugin is actively supported by its registered maintainers
>
> To support a Fuel plugin, we need to answer two fundamental questions:
> Can we? Should we?
>
> I think the minimum requirements to say "yes" to both are:
>
> a) All of the plugin's source code is explicitly licensed under an
>OSI-approved license;
>
> b) The plugin source code repository does not contain binary artefacts
>such as RPM packages or ISO images (*);
>
> c) The plugin is registered in DriverLog;
>
> d) Plugin maintainers listed in DriverLog have confirmed the intent to
>support the plugin;
>
> e) Plugin repository on review.openstack.org has a voting CI job that is
>passing with the latest or, at least, previous major release of Fuel.
>
> f) All deviations from the OpenStack development process (alternative
>issue trackers, mailing lists, etc.) are documented in the plugin's
>README file.
>
> *  Aside from purely

Re: [openstack-dev] [Fuel] Task Based Deployment Is at Least Twice Faster

2016-02-16 Thread Evgeniy L

That is awesome, happy to finally see it enabled!

On Mon, Feb 15, 2016 at 9:32 PM, Anastasia Urlapova 
wrote:

> Aleksey, great news!
>
> On Mon, Feb 15, 2016 at 7:36 PM, Alexey Shtokolov  > wrote:
>
>> Fuelers,
>>
>> Task based deployment engine has been enabled in master (Fuel 9.0) by
>> default [0]
>>
>> [0] - https://review.openstack.org/#/c/273693/
>>
>> WBR, Alexey Shtokolov
>>
>> 2016-02-09 21:57 GMT+03:00 Vladimir Kuklin :
>>
>>> Folks
>>>
>>> It seems that docker removal spoilt our celebration a bit. Here is a bug
>>> link https://bugs.launchpad.net/fuel/+bug/1543720 . Fix is trivial, but
>>> will postpone swarm run for another day. Nevertheless, it seems to be the
>>> only issue affecting our ability to use TBD.
>>>
>>> Stay tuned!
>>>
>>> On Tue, Feb 9, 2016 at 2:26 PM, Igor Kalnitsky 
>>> wrote:
>>>
 > I've run BVT more than 100 times, it works,

 You run it some time ago. There were a lot of opportunities to
 introduce regression in both Nailgun and tasks of Fuel Library. ;)

 > We are going to run a swarm test today against the ISO with enabled
 task-based deployment

 So there will be a custom ISO, correct? If so, it works for me and
 I'll wait for its result.

 On Tue, Feb 9, 2016 at 1:17 PM, Alexey Shtokolov
  wrote:
 > Igor,
 >
 > We are going to run a swarm test today against the ISO with enabled
 > task-based deployment, than check results and merge changes tomorrow.
 > I've run BVT more than 100 times, it works, but I would like to check
 more
 > deployment cases.
 > And I guess it should be easy to troubleshoot if docker-related and
 > task-based related changes will be separated by a few days.
 >
 > 2016-02-09 13:39 GMT+03:00 Igor Kalnitsky :
 >>
 >> Well, I'm going to build a new ISO and run BVT. As soon as they are
 >> green, I'm going to approve the change.
 >>
 >> On Tue, Feb 9, 2016 at 12:32 PM, Bogdan Dobrelya <
 bdobre...@mirantis.com>
 >> wrote:
 >> > On 08.02.2016 17:05, Igor Kalnitsky wrote:
 >> >> Hey Fuelers,
 >> >>
 >> >> When we are going to enable it? I think since HCF is passed for
 >> >> stable/8.0, it's time to enable task-based deployment for master
 >> >> branch.
 >> >>
 >> >> Opinion?
 >> >
 >> > This must be done for the 9.0, IMHO.
 >> >
 >> >>
 >> >> - Igor
 >> >>
 >> >> On Wed, Feb 3, 2016 at 12:31 PM, Bogdan Dobrelya
 >> >>  wrote:
 >> >>> On 02.02.2016 17:35, Alexey Shtokolov wrote:
 >>  Hi Fuelers!
 >> 
 >>  As you may be aware, since [0] Fuel has implemented a new
 >>  orchestration
 >>  engine [1]
 >>  We switched the deployment paradigm from role-based (aka
 granular) to
 >>  task-based and now Fuel can deploy all nodes simultaneously
 using
 >>  cross-node dependencies between deployment tasks.
 >> >>>
 >> >>> That is great news! Please do not forget about docs updates as
 well.
 >> >>> Those docs are always forgotten like poor orphans... I submitted
 a
 >> >>> patch
 >> >>> [0] to MOS docs, please review and add more details, if
 possible, for
 >> >>> plugins impact as well.
 >> >>>
 >> >>> [0] https://review.fuel-infra.org/#/c/16509/
 >> >>>
 >> 
 >>  This feature is experimental in Fuel 8.0 and will be enabled by
 >>  default
 >>  for Fuel 9.0
 >> 
 >>  Allow me to show you the results. We made some benchmarks on
 our bare
 >>  metal lab [2]
 >> 
 >>  Case #1. 3 controllers + 7 computes w/ ceph.
 >>  Task-based deployment takes *~38* minutes vs *~1h15m* for
 granular
 >>  (*~2*
 >>  times faster)
 >>  Here and below the deployment time is average time for 10 runs
 >> 
 >>  Case #2. 3 controllers + 3 mongodb + 4 computes w/ ceph.
 >>  Task-based deployment takes *~41* minutes vs *~1h32m* for
 granular
 >>  (*~2.24* times faster)
 >> 
 >> 
 >> 
 >>  Also we took measurements for Fuel CI test cases. Standard BVT
 >>  (Master
 >>  node + 3 controllers + 3 computes w/ ceph. All are in qemu VMs
 on one
 >>  host)
 >> 
 >>  Fuel CI slaves with *4 *cores *~1.1* times faster
 >>  In case of 4 cores for 7 VMs they are fighting for CPU
 resources and
 >>  it
 >>  marginalizes the gain of task-based deployment
 >> 
 >>  Fuel CI slaves with *6* cores *~1.6* times faster
 >> 
 >>  Fuel CI slaves with *12* cores *~1.7* times faster
 >> >>>
 >> >>> These are really outstanding results!
 >> >>>

[openstack-dev] [Bareon] Weekly update

2016-02-15 Thread Evgeniy L

Hi,

After the discussion with some folks, we agreed that it might be useful for
the community to start sending weekly updates on what Bareon team is
working on and what is our progress.

So here is a first weekly update from Bareon team.

1. Data pipelines for Nailgun integration (changing provisioning/deployment
data with extensions), previous week we've had an active discussion, mostly
agreed on implementation, both spec [0] and code [1] are on review.
2. Bugs debugging/reviewing/fixing [2]-[8]
3. Due to bugs not much progress on pluggable do actions this week [9]
4. Dynamic allocation, no progress [10]
5. Engineers from Cray team have 6 features [11] for upstream Bareon,
currently we are trying to figure out what are the best ways to land these
changes.

Thanks,

[0] https://review.openstack.org/#/c/274653/
[1] https://review.openstack.org/#/c/272977/

[2] https://bugs.launchpad.net/fuel/+bug/1538645
[3] https://bugs.launchpad.net/fuel/+bug/1543063
[4] https://bugs.launchpad.net/fuel/+bug/1544818
[5] https://bugs.launchpad.net/fuel/+bug/1544816
[6] https://bugs.launchpad.net/fuel/+bug/1543240
[7] https://bugs.launchpad.net/fuel/+bug/1543233
[8] https://bugs.launchpad.net/fuel/+bug/1543221

[9] https://blueprints.launchpad.net/bareon/+spec/pluggable-do-actions
[10] https://blueprints.launchpad.net/bareon/+spec/dynamic-allocation
[11]
https://review.openstack.org/#/q/project:openstack/bareon-specs+owner:%22Max+Lobur+%253Cmax_lobur%2540outlook.com%253E%22
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins] Multi release packages

2016-02-12 Thread Evgeniy L

>> We have package format <=4.0 where all files have fixed names and
locations. This is the defaults.

The thing is for 5.0 there should be no default, those fields from now on
should be specified explicitly.

>> Igor want to provide multi-package

I'm not familiar with this idea, could you please clarify what
multi-package is?

Thanks,

On Fri, Feb 12, 2016 at 2:57 PM, Ilya Kutukov <ikutu...@mirantis.com> wrote:

>
>
> On Fri, Feb 12, 2016 at 2:03 PM, Evgeniy L <e...@mirantis.com> wrote:
>
>> Ilya,
>>
>> What do you mean by "default"? From the data format I see that we don't
>> "override defaults" we specify the data for specific release, the way it
>> was always done for deployment scripts and repositories.
>>
>>
> We have package format <=4.0 where all files have fixed names and
> locations. This is the defaults.
>
> 1. The maintenance team wants ability to specify folder instead plugin
> configuration files so there should be ability to change this paths to
> define a folder or other non-standard location. Yes, plugin developer could
> have whatever source structure and then translate it to the file structure
> required for the FPB with scripts or build system, but ability to split
> e.g. tasks files looks useful for me.
>
> 2. Igor want to provide multi-package, so, according to spec, this custom
> release-specific paths to other package files could be specified in release
> records.
>
> I don't see any reasons to complicate format even more and to have some
>> things which are related to the deployment specified in the root and some
>> in specific release.
>>
>> There is consistent mechanism to specify such kind of things, lets just
>> use it.
>>
>> Thanks,
>>
>> On Fri, Feb 12, 2016 at 1:24 PM, Ilya Kutukov <ikutu...@mirantis.com>
>> wrote:
>>
>>>
>>>
>>> On Fri, Feb 12, 2016 at 11:47 AM, Evgeniy L <e...@mirantis.com> wrote:
>>>
>>>> Ilya,
>>>>
>>>> >> My opinion that i've seen no example of multiple software of plugins
>>>> versions shipped in one package or other form of bundle. Its not a common
>>>> practice.
>>>>
>>>> With plugins we extend Fuel capabilities, which supports multiple
>>>> operating system releases, so it's absolutely natural to extend multiple
>>>> releases at the same time.
>>>>
>>>>
>>> I just warning against idea when to merge content of several plugin
>>> distributions in one bundle. But it's ok for plugin code to support
>>> multiple releases one or another way.
>>>
>>>
>>>
>>>> >> Anyway we need to provide ability to override paths in manifest
>>>> (metadata.yaml).
>>>>
>>>> Could you please provide more information on that? I'm not sure if I
>>>> understand your solution.
>>>>
>>>>
>>> https://review.openstack.org/#/c/271417/5/specs/9.0/plugins-v5.rst L150
>>> and further
>>> We are overriding default path with special per-release path attributes.
>>> The question is to use per-release way described in spec or don't bother
>>> and specify this overrides only in metadata.yaml root.
>>>
>>>
>>>> Also I'm not sure what we are arguing about, if plugin developer (or
>>>> certification process of some company) requires to have plugin per release,
>>>> it's *very easy* and straight forward to do it even now *without any*
>>>> changes.
>>>>
>>>> If plugin developer wants to deliver plugin for CentOS, Ubuntu, RH etc,
>>>> let them do it, and again when we get full support
>>>> of multi-version environments it's going to be even more crucial for UX to
>>>> have a single plugin with multi-release support.
>>>>
>>>>
>>> Thanks,
>>>>
>>>> On Thu, Feb 11, 2016 at 9:35 PM, Ilya Kutukov <ikutu...@mirantis.com>
>>>> wrote:
>>>>
>>>>> My opinion that i've seen no example of multiple software of plugins
>>>>> versions shipped in one package or other form of bundle. Its not a common
>>>>> practice.
>>>>>
>>>>> Anyway we need to provide ability to override paths in manifest
>>>>> (metadata.yaml).
>>>>>
>>>>> So the plugin developers could use this approaches to provide multiple
>>>>> versions support:
>>>>>
>>>>> * tasks logic (do the plug

Re: [openstack-dev] [Fuel][Plugins] Multi release packages

2016-02-12 Thread Evgeniy L

Ilya,

What do you mean by "default"? From the data format I see that we don't
"override defaults" we specify the data for specific release, the way it
was always done for deployment scripts and repositories.

I don't see any reasons to complicate format even more and to have some
things which are related to the deployment specified in the root and some
in specific release.

There is consistent mechanism to specify such kind of things, lets just use
it.

Thanks,

On Fri, Feb 12, 2016 at 1:24 PM, Ilya Kutukov <ikutu...@mirantis.com> wrote:

>
>
> On Fri, Feb 12, 2016 at 11:47 AM, Evgeniy L <e...@mirantis.com> wrote:
>
>> Ilya,
>>
>> >> My opinion that i've seen no example of multiple software of plugins
>> versions shipped in one package or other form of bundle. Its not a common
>> practice.
>>
>> With plugins we extend Fuel capabilities, which supports multiple
>> operating system releases, so it's absolutely natural to extend multiple
>> releases at the same time.
>>
>>
> I just warning against idea when to merge content of several plugin
> distributions in one bundle. But it's ok for plugin code to support
> multiple releases one or another way.
>
>
>
>> >> Anyway we need to provide ability to override paths in manifest
>> (metadata.yaml).
>>
>> Could you please provide more information on that? I'm not sure if I
>> understand your solution.
>>
>>
> https://review.openstack.org/#/c/271417/5/specs/9.0/plugins-v5.rst L150
> and further
> We are overriding default path with special per-release path attributes.
> The question is to use per-release way described in spec or don't bother
> and specify this overrides only in metadata.yaml root.
>
>
>> Also I'm not sure what we are arguing about, if plugin developer (or
>> certification process of some company) requires to have plugin per release,
>> it's *very easy* and straight forward to do it even now *without any*
>> changes.
>>
>> If plugin developer wants to deliver plugin for CentOS, Ubuntu, RH etc,
>> let them do it, and again when we get full support
>> of multi-version environments it's going to be even more crucial for UX to
>> have a single plugin with multi-release support.
>>
>>
> Thanks,
>>
>> On Thu, Feb 11, 2016 at 9:35 PM, Ilya Kutukov <ikutu...@mirantis.com>
>> wrote:
>>
>>> My opinion that i've seen no example of multiple software of plugins
>>> versions shipped in one package or other form of bundle. Its not a common
>>> practice.
>>>
>>> Anyway we need to provide ability to override paths in manifest
>>> (metadata.yaml).
>>>
>>> So the plugin developers could use this approaches to provide multiple
>>> versions support:
>>>
>>> * tasks logic (do the plugin developers have access to current release
>>> info?)
>>> * hooks in pre-build process. Its not a big deal to preprocess source
>>> folder to build different packages with scripts that adding or removing
>>> some files or replacing some paths.
>>> * and, perhaps, logic anchors with YACL or other DSL in tasks
>>> dependancies if this functionality will be added this in theory could allow
>>> to use or not to use some graph parts depending on release.
>>>
>>> I think its already better than nothing and more flexible than any
>>> standardised approach.
>>>
>>> On Thu, Feb 11, 2016 at 6:31 PM, Simon Pasquier <spasqu...@mirantis.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> On Thu, Feb 11, 2016 at 11:46 AM, Igor Kalnitsky <
>>>> ikalnit...@mirantis.com> wrote:
>>>>
>>>>> Hey folks,
>>>>>
>>>>> The original idea is to provide a way to build plugin that are
>>>>> compatible with few releases. It makes sense to me, cause it looks
>>>>> awful if you need to maintain different branches for different Fuel
>>>>> releases and there's no difference in the sources. In that case, each
>>>>> bugfix to deployment scripts requires:
>>>>>
>>>>> * backport bugfix to other branches (N backports)
>>>>> * build new packages for supported releases (N builds)
>>>>> * release new packages (N releases)
>>>>>
>>>>> It's somehow.. annoying.
>>>>>
>>>>
>>>> A big +1 on Igor's remark. I've already expressed it in another thread
>>>> but it should be expected that plugin developers

Re: [openstack-dev] [Fuel][Plugins] Multi release packages

2016-02-12 Thread Evgeniy L

Ilya,

>> My opinion that i've seen no example of multiple software of plugins
versions shipped in one package or other form of bundle. Its not a common
practice.

With plugins we extend Fuel capabilities, which supports multiple operating
system releases, so it's absolutely natural to extend multiple releases at
the same time.

>> Anyway we need to provide ability to override paths in manifest
(metadata.yaml).

Could you please provide more information on that? I'm not sure if I
understand your solution.

Also I'm not sure what we are arguing about, if plugin developer (or
certification process of some company) requires to have plugin per release,
it's *very easy* and straight forward to do it even now *without any*
changes.

If plugin developer wants to deliver plugin for CentOS, Ubuntu, RH etc, let
them do it, and again when we get full support
of multi-version environments it's going to be even more crucial for UX to
have a single plugin with multi-release support.

Thanks,

On Thu, Feb 11, 2016 at 9:35 PM, Ilya Kutukov  wrote:

> My opinion that i've seen no example of multiple software of plugins
> versions shipped in one package or other form of bundle. Its not a common
> practice.
>
> Anyway we need to provide ability to override paths in manifest
> (metadata.yaml).
>
> So the plugin developers could use this approaches to provide multiple
> versions support:
>
> * tasks logic (do the plugin developers have access to current release
> info?)
> * hooks in pre-build process. Its not a big deal to preprocess source
> folder to build different packages with scripts that adding or removing
> some files or replacing some paths.
> * and, perhaps, logic anchors with YACL or other DSL in tasks dependancies
> if this functionality will be added this in theory could allow to use or
> not to use some graph parts depending on release.
>
> I think its already better than nothing and more flexible than any
> standardised approach.
>
> On Thu, Feb 11, 2016 at 6:31 PM, Simon Pasquier 
> wrote:
>
>> Hi,
>>
>> On Thu, Feb 11, 2016 at 11:46 AM, Igor Kalnitsky > > wrote:
>>
>>> Hey folks,
>>>
>>> The original idea is to provide a way to build plugin that are
>>> compatible with few releases. It makes sense to me, cause it looks
>>> awful if you need to maintain different branches for different Fuel
>>> releases and there's no difference in the sources. In that case, each
>>> bugfix to deployment scripts requires:
>>>
>>> * backport bugfix to other branches (N backports)
>>> * build new packages for supported releases (N builds)
>>> * release new packages (N releases)
>>>
>>> It's somehow.. annoying.
>>>
>>
>> A big +1 on Igor's remark. I've already expressed it in another thread
>> but it should be expected that plugin developers want to support 2
>> consecutive versions of Fuel for a given version of their plugin.
>> That being said, I've never had issues to do it with the current plugin
>> framework. Except when Fuel breaks the backward compatibility but it's
>> another story...
>>
>> Simon
>>
>>
>>>
>>> However, I starting agree that having all-in-one RPM when deployment
>>> scripts are different, tasks are different, roles/volumes are
>>> different, probably isn't a good idea. It basically means that your
>>> sources are completely different, and that means you have different
>>> implementations of the same plugin. In that case, in order to avoid
>>> mess in source tree, it'd be better to separate such implementations
>>> on VCS level.
>>>
>>> But I'd like to hear more opinion from plugin developers.
>>>
>>> - Igor
>>>
>>> On Thu, Feb 11, 2016 at 9:16 AM, Bulat Gaifullin
>>>  wrote:
>>> > I agree with Stas, one rpm - one version.
>>> >
>>> > But plugin builder allows to specify several releases as compatible.
>>> The
>>> > deployment tasks and repositories can be specified per release, at the
>>> same
>>> > time the deployment graph is one for all releases.
>>> > Currently it looks like half-implemented feature.  Can we drop this
>>> feature?
>>> > or should we finish implementation of this feature.
>>> >
>>> >
>>> > Regards,
>>> > Bulat Gaifullin
>>> > Mirantis Inc.
>>> >
>>> >
>>> >
>>> > On 11 Feb 2016, at 02:41, Andrew Woodward  wrote:
>>> >
>>> >
>>> >
>>> > On Wed, Feb 10, 2016 at 2:23 PM Dmitry Borodaenko <
>>> dborodae...@mirantis.com>
>>> > wrote:
>>> >>
>>> >> +1 to Stas, supplanting VCS branches with code duplication is a path
>>> to
>>> >> madness and despair. The dubious benefits of a cross-release backwards
>>> >> compatible plugin binary are not worth the code and infra technical
>>> debt
>>> >> that such approach would accrue over time.
>>> >
>>> >
>>> > Supporting multiple fuel releases will likely result in madness as
>>> > discussed, however as we look to support multiple OpenStack releases
>>> from
>>> > the same version of fuel, this methodology becomes much more important.
>>> >

Re: [openstack-dev] [Fuel][Plugins] Multi release packages

2016-02-11 Thread Evgeniy L

Sorry for the typo "s/I can shade more light/I can shed more light/"

On Thu, Feb 11, 2016 at 1:51 PM, Evgeniy L <e...@mirantis.com> wrote:

> Hi,
>
> As an author of this part of pluggable architecture I can shade more light
> on why it was implemented this way and why it's valuable to continue
> supporting multi-release feature.
>
> At the time it was implemented Fuel officially was supporting both Ubuntu
> and CentOS, in order to simplify plugins distribution development and
> support,
> plugin developer was able to create a single plugin which supports both
> operating systems, deployment scripts in the most cases support both
> operating systems, if they are not, there is enough flexibility to specify
> different set of deployment scripts and repositories.
>
> We don't force plugin developer to support all range of releases in a
> single
> plugin, it's up to plugin developer to decide what mechanism is better for
> specific plugin, I'm strongly against of making "single release a.k.a os"
> plugins.
>
> Also suggested change will dramatically complicate distribution when we
> get heterogeneous environments (different operating system in a single
> environment).
>
> Bulat is right that there are issues, but those issues has to be fixed, and
> there is straightforward way to do it.
>
> To summarise, I don't think that we should force the developer to follow
> specific release schema, let the developer decide.
>
> Thanks,
>
>
> On Thu, Feb 11, 2016 at 10:16 AM, Bulat Gaifullin <bgaiful...@mirantis.com
> > wrote:
>
>> I agree with Stas, one rpm - one version.
>>
>> But plugin builder allows to specify several releases as compatible. The
>> deployment tasks and repositories can be specified per release, at the same
>> time the deployment graph is one for all releases.
>> Currently it looks like half-implemented feature.  Can we drop this
>> feature? or should we finish implementation of this feature.
>>
>>
>> Regards,
>> Bulat Gaifullin
>> Mirantis Inc.
>>
>>
>>
>> On 11 Feb 2016, at 02:41, Andrew Woodward <xar...@gmail.com> wrote:
>>
>>
>>
>> On Wed, Feb 10, 2016 at 2:23 PM Dmitry Borodaenko <
>> dborodae...@mirantis.com> wrote:
>>
>>> +1 to Stas, supplanting VCS branches with code duplication is a path to
>>> madness and despair. The dubious benefits of a cross-release backwards
>>> compatible plugin binary are not worth the code and infra technical debt
>>> that such approach would accrue over time.
>>>
>>
>> Supporting multiple fuel releases will likely result in madness as
>> discussed, however as we look to support multiple OpenStack releases from
>> the same version of fuel, this methodology becomes much more important.
>>
>>
>>> On Wed, Feb 10, 2016 at 07:36:30PM +0300, Stanislaw Bogatkin wrote:
>>> > It changes mostly nothing for case of furious plugin development when
>>> big
>>> > parts of code changed from one release to another.
>>> >
>>> > You will have 6 different deployment_tasks directories and 30 a little
>>> bit
>>> > different files in root directory of plugin. Also you forgot about
>>> > repositories directory (+6 at least), pre_build hooks (also 6) and so
>>> on.
>>> > It will look as hell after just 3 years of development.
>>> >
>>> > Also I can't imagine how to deal with plugin licensing if you have
>>> Apache
>>> > for liberty but BSD for mitaka release, for example.
>>> >
>>> > Much easier way to develop a plugin is to keep it's source in VCS like
>>> Git
>>> > and just make a branches for every fuel release. It will give us
>>> > opportunity to not store a bunch of similar but a little bit different
>>> > files in repo. There is no reason to drag all different versions of
>>> code
>>> > for specific release.
>>> >
>>> >
>>> > On other hand there is a pros - your plugin can survive after upgrade
>>> if it
>>> > supports new release, no changes needed here.
>>> >
>>> > On Wed, Feb 10, 2016 at 4:04 PM, Alexey Shtokolov <
>>> ashtoko...@mirantis.com>
>>> > wrote:
>>> >
>>> > > Fuelers,
>>> > >
>>> > > We are discussing the idea to extend the multi release packages for
>>> > > plugins.
>>> > >
>>> > > Fuel plugin builder (FPB) can create one rpm-package for all

Re: [openstack-dev] [Fuel] Task Based Deployment Is at Least Twice Faster

2016-02-09 Thread Evgeniy L

+1

On Mon, Feb 8, 2016 at 7:58 PM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> +1 to enable it ASAP.
>
> It will also affect our deployment tests (~1 hour vs. ~2.5 hours).
>
> Vladimir Kozhukalov
>
> On Mon, Feb 8, 2016 at 7:35 PM, Bulat Gaifullin 
> wrote:
>
>> +1.
>>
>> Regards,
>> Bulat Gaifullin
>> Mirantis Inc.
>>
>>
>>
>> > On 08 Feb 2016, at 19:05, Igor Kalnitsky 
>> wrote:
>> >
>> > Hey Fuelers,
>> >
>> > When we are going to enable it? I think since HCF is passed for
>> > stable/8.0, it's time to enable task-based deployment for master
>> > branch.
>> >
>> > Opinion?
>> >
>> > - Igor
>> >
>> > On Wed, Feb 3, 2016 at 12:31 PM, Bogdan Dobrelya <
>> bdobre...@mirantis.com> wrote:
>> >> On 02.02.2016 17:35, Alexey Shtokolov wrote:
>> >>> Hi Fuelers!
>> >>>
>> >>> As you may be aware, since [0] Fuel has implemented a new
>> orchestration
>> >>> engine [1]
>> >>> We switched the deployment paradigm from role-based (aka granular) to
>> >>> task-based and now Fuel can deploy all nodes simultaneously using
>> >>> cross-node dependencies between deployment tasks.
>> >>
>> >> That is great news! Please do not forget about docs updates as well.
>> >> Those docs are always forgotten like poor orphans... I submitted a
>> patch
>> >> [0] to MOS docs, please review and add more details, if possible, for
>> >> plugins impact as well.
>> >>
>> >> [0] https://review.fuel-infra.org/#/c/16509/
>> >>
>> >>>
>> >>> This feature is experimental in Fuel 8.0 and will be enabled by
>> default
>> >>> for Fuel 9.0
>> >>>
>> >>> Allow me to show you the results. We made some benchmarks on our bare
>> >>> metal lab [2]
>> >>>
>> >>> Case #1. 3 controllers + 7 computes w/ ceph.
>> >>> Task-based deployment takes *~38* minutes vs *~1h15m* for granular
>> (*~2*
>> >>> times faster)
>> >>> Here and below the deployment time is average time for 10 runs
>> >>>
>> >>> Case #2. 3 controllers + 3 mongodb + 4 computes w/ ceph.
>> >>> Task-based deployment takes *~41* minutes vs *~1h32m* for granular
>> >>> (*~2.24* times faster)
>> >>>
>> >>>
>> >>>
>> >>> Also we took measurements for Fuel CI test cases. Standard BVT (Master
>> >>> node + 3 controllers + 3 computes w/ ceph. All are in qemu VMs on one
>> host)
>> >>>
>> >>> Fuel CI slaves with *4 *cores *~1.1* times faster
>> >>> In case of 4 cores for 7 VMs they are fighting for CPU resources and
>> it
>> >>> marginalizes the gain of task-based deployment
>> >>>
>> >>> Fuel CI slaves with *6* cores *~1.6* times faster
>> >>>
>> >>> Fuel CI slaves with *12* cores *~1.7* times faster
>> >>
>> >> These are really outstanding results!
>> >> (tl;dr)
>> >> I believe the next step may be to leverage the "external install & svc
>> >> management" feature (example [1]) of the Liberty release (7.0.0) of
>> >> Puppet-Openstack (PO) modules. So we could use separate concurrent
>> >> cross-depends based tasks *within a single node* as well, like:
>> >> - task: install_all_packages - a singleton task for a node,
>> >> - task: [configure_x, for each x] - concurrent for a node,
>> >> - task: [manage_service_x, for each x] - some may be concurrent for a
>> >> node, while another shall be serialized.
>> >>
>> >> So, one might use the "--tags" separator for concurrent puppet runs to
>> >> make things go even faster, for example:
>> >>
>> >> # cat test.pp
>> >> notify
>> >> {"A": tag => "a" }
>> >> notify
>> >> {"B": tag => "b" }
>> >>
>> >> # puppet apply test.pp
>> >> Notice: A
>> >> Notice: /Stage[main]/Main/Notify[A]/message: defined 'message' as 'A'
>> >> Notice: B
>> >> Notice: /Stage[main]/Main/Notify[B]/message: defined 'message' as 'B'
>> >>
>> >> # puppet apply test.pp --tags a
>> >> Notice: A
>> >> Notice: /Stage[main]/Main/Notify[A]/message: defined 'message' as 'A'
>> >>
>> >> # puppet apply test.pp --tags a & puppet apply test.pp --tags b
>> >> Notice: B
>> >> Notice: /Stage[main]/Main/Notify[B]/message: defined 'message' as 'B'
>> >> Notice: A
>> >> Notice: /Stage[main]/Main/Notify[A]/message: defined 'message' as 'A'
>> >>
>> >> Which is supposed to be faster, although not for this example.
>> >>
>> >> [1] https://review.openstack.org/#/c/216926/3/manifests/init.pp
>> >>
>> >>>
>> >>> You can see additional information and charts in the presentation [3].
>> >>>
>> >>> [0]
>> >>> -
>> http://lists.openstack.org/pipermail/openstack-dev/2015-December/082093.html
>> >>> [1]
>> >>> -
>> https://specs.openstack.org/openstack/fuel-specs/specs/8.0/task-based-deployment-mvp.html
>> >>> [2] -  3 x HP ProLiant DL360p Gen8 (XeonE5 6 cores/64GB/SSD)  + 7 x HP
>> >>> ProLiant DL320p Gen8 (XeonE3 4 cores/8-16GB/HDD)
>> >>> [3] -
>> >>>
>> https://docs.google.com/presentation/d/1jZCFZlXHs_VhjtVYS2VuWgdxge5Q6sOMLz4bRLuw7YE
>> >>>
>> >>> ---
>> >>> WBR, Alexey Shtokolov
>> >>>
>> >>>
>> >>>
>> __
>> >>> OpenStack Development Mailing List (not for usage questions)
>> >>>

Re: [openstack-dev] [Fuel] Nominate Fedor Zhadaev for the fuel-menu-core team

2016-02-08 Thread Evgeniy L

+1

On Mon, Feb 8, 2016 at 12:54 PM, Igor Kalnitsky 
wrote:

> Hey Fuelers,
>
> I'd like to nominate Fedor Zhadaev for the fuel-menu-core team.
> Fedor's doing good review with detailed feedback [1], and has
> contributes over 20 patches during Mitaka release cycle [2].
>
> Fuel Cores, please reply back with +1/-1.
>
> - igor
>
> [1] http://stackalytics.com/?module=fuel-menu=mitaka
> [2]
> http://stackalytics.com/?module=fuel-menu=mitaka_id=fzhadaev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Bareon][Fuel] Best practises on syncing patches between repositories

2016-02-08 Thread Evgeniy L

Hi,

Some time ago we started Bareon project [1], and now we have some fixes
landed to
fuel-agent only, the question is what are the best practises on keeping two
repos in sync
with possibility to resolve conflicts manually? Cherry-picking patches
manually doesn't look
like the most error prone solution, are there any scripts written to make
sure that repos are
in one way sync (fuel-agent -> Bareon)?

Thanks,

[1]
http://lists.openstack.org/pipermail/openstack-dev/2015-December/082397.html
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins] question on the is_hotpluggable feature

2016-02-05 Thread Evgeniy L

Simon,

>> Any plan to have a nicer experience in future Fuel releases?

I haven't heard about any plans on improvements for that, but management
team should know better whether it's on roadmap or not.

Thanks,

On Fri, Feb 5, 2016 at 1:52 PM, Simon Pasquier <spasqu...@mirantis.com>
wrote:

> Thanks Evgeniy.
>
> On Fri, Feb 5, 2016 at 11:07 AM, Evgeniy L <e...@mirantis.com> wrote:
>
>> Hi Simon,
>>
>> As far as I know it's expected behaviour (at least for the current
>> release), and it's expected that user reruns deployment on required nodes
>> using fuel cli, in order to install plugin on a live environment.
>>
>
> Ok. For the record, this means running this command for every node that is
> already deployed:
> $ fuel node --node-id  --deploy
>
> Any plan to have a nicer experience in future Fuel releases?
>
>
>> It depends on specific role, but "update_required" field may help you, it
>> can be added to role description, Fuel reruns deployment on nodes with
>> roles, which are specified in the list, if new node with the role is added
>> to the environment.
>>
>
> Nope, it doesn't work for me since it should run for *all* the nodes,
> irrespective of their roles. AFAIK update_required doesn't support '*'.
>
>
>>
>> Thanks,
>>
>> [1]
>> https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/fixtures/openstack.yaml#L16-L18
>>
>> On Fri, Feb 5, 2016 at 12:53 PM, Simon Pasquier <spasqu...@mirantis.com>
>> wrote:
>>
>>> Hi,
>>> I'm testing the ability to install Fuel plugins in a an environment that
>>> is already deployed.
>>> My starting environment is quite simple: 1 controller + 1 compute. After
>>> the initial deployment, I've installed the 4 LMA plugins:
>>> - LMA collector
>>> - Elasticsearch-Kibana [*]
>>> - InfluxDB-Grafana [*]
>>> - Infrastructure Alerting [*]
>>> [*] adds a new role
>>> Of course, all plugins have "is_hotpluggable: true" in their metadata
>>> definition.
>>> My expectation is that I can add a new node with the new roles and that
>>> the LMA collector tasks are executed for all 3 nodes. So I've added the new
>>> node and click the "Deploy changes" button. My re-deployment runs fine but
>>> I notice that the plugins aren't installed on the existing nodes (eg
>>> /etc/fuel/plugins/...) so there is no way that the plugins tasks can be
>>> executed on already deployed nodes... Is this a known limitation? Am I
>>> missing something?
>>> Best regards,
>>> Simon
>>>
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins] question on the is_hotpluggable feature

2016-02-05 Thread Evgeniy L

Hi Simon,

As far as I know it's expected behaviour (at least for the current
release), and it's expected that user reruns deployment on required nodes
using fuel cli, in order to install plugin on a live environment.
It depends on specific role, but "update_required" field may help you, it
can be added to role description, Fuel reruns deployment on nodes with
roles, which are specified in the list, if new node with the role is added
to the environment.

Thanks,

[1]
https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/fixtures/openstack.yaml#L16-L18

On Fri, Feb 5, 2016 at 12:53 PM, Simon Pasquier 
wrote:

> Hi,
> I'm testing the ability to install Fuel plugins in a an environment that
> is already deployed.
> My starting environment is quite simple: 1 controller + 1 compute. After
> the initial deployment, I've installed the 4 LMA plugins:
> - LMA collector
> - Elasticsearch-Kibana [*]
> - InfluxDB-Grafana [*]
> - Infrastructure Alerting [*]
> [*] adds a new role
> Of course, all plugins have "is_hotpluggable: true" in their metadata
> definition.
> My expectation is that I can add a new node with the new roles and that
> the LMA collector tasks are executed for all 3 nodes. So I've added the new
> node and click the "Deploy changes" button. My re-deployment runs fine but
> I notice that the plugins aren't installed on the existing nodes (eg
> /etc/fuel/plugins/...) so there is no way that the plugins tasks can be
> executed on already deployed nodes... Is this a known limitation? Am I
> missing something?
> Best regards,
> Simon
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Bareon][Fuel] Dynamic allocation algorithm

2016-01-14 Thread Evgeniy L

Hi,

In addition I've generated several examples in order to show how current
prototype allocates the volumes [0].

Thanks,

[0] http://bareon-allocator.readthedocs.org/en/latest/examples.html


On Wed, Jan 13, 2016 at 2:14 PM, Evgeniy L <e...@mirantis.com> wrote:

> Hi Artur,
>
> You are correct, we probably may consider using bytes instead of megabytes.
>
> Regarding to question "ssd vs hdd", user can describe which space is better
> to allocate on ssd and which is better on hdd, the mechanism is completely
> data driven,
> it can be done using "best_with_disks" [0], in fact it covers much more
> cases,
> since user can build sets of disks where space can be allocated based on
> any
> parameter of HW which discovery can provide.
>
> Thanks,
>
> [0]
> http://bareon-allocator.readthedocs.org/en/latest/architecture.html#best-with-disks
>
> On Wed, Jan 13, 2016 at 1:56 PM, Artur Svechnikov <
> asvechni...@mirantis.com> wrote:
>
>> Hi.
>>
>> Very good documentation. For Integer Solution you can use bytes instead
>> of megabytes. Hence N bytes will be unallocated in the worst case.
>>
>> I didn't find solution for problem:
>>
>>- Don’t allocate a single volume on ssd and hdd
>>
>>
>> Best regards,
>> Svechnikov Artur
>>
>> On Tue, Jan 12, 2016 at 9:37 PM, Evgeniy L <e...@mirantis.com> wrote:
>>
>>> Hi,
>>>
>>> For the last several weeks I've been working on algorithm (and prototype)
>>> for dynamic allocation of volumes on disks.
>>>
>>> I have some results [0] and would like to ask you to review it and
>>> provide
>>> some feedback.
>>>
>>> Our plan is to implement it as an external driver for Bareon [1].
>>>
>>> Thanks,
>>>
>>> [0] http://bareon-allocator.readthedocs.org/en/latest/architecture.html
>>> [1] https://wiki.openstack.org/wiki/Bareon
>>>
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Bareon][Fuel] Dynamic allocation algorithm

2016-01-13 Thread Evgeniy L

Hi Artur,

You are correct, we probably may consider using bytes instead of megabytes.

Regarding to question "ssd vs hdd", user can describe which space is better
to allocate on ssd and which is better on hdd, the mechanism is completely
data driven,
it can be done using "best_with_disks" [0], in fact it covers much more
cases,
since user can build sets of disks where space can be allocated based on any
parameter of HW which discovery can provide.

Thanks,

[0]
http://bareon-allocator.readthedocs.org/en/latest/architecture.html#best-with-disks

On Wed, Jan 13, 2016 at 1:56 PM, Artur Svechnikov <asvechni...@mirantis.com>
wrote:

> Hi.
>
> Very good documentation. For Integer Solution you can use bytes instead of
> megabytes. Hence N bytes will be unallocated in the worst case.
>
> I didn't find solution for problem:
>
>- Don’t allocate a single volume on ssd and hdd
>
>
> Best regards,
> Svechnikov Artur
>
> On Tue, Jan 12, 2016 at 9:37 PM, Evgeniy L <e...@mirantis.com> wrote:
>
>> Hi,
>>
>> For the last several weeks I've been working on algorithm (and prototype)
>> for dynamic allocation of volumes on disks.
>>
>> I have some results [0] and would like to ask you to review it and provide
>> some feedback.
>>
>> Our plan is to implement it as an external driver for Bareon [1].
>>
>> Thanks,
>>
>> [0] http://bareon-allocator.readthedocs.org/en/latest/architecture.html
>> [1] https://wiki.openstack.org/wiki/Bareon
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Bareon][Fuel] Dynamic allocation algorithm

2016-01-12 Thread Evgeniy L

Hi,

For the last several weeks I've been working on algorithm (and prototype)
for dynamic allocation of volumes on disks.

I have some results [0] and would like to ask you to review it and provide
some feedback.

Our plan is to implement it as an external driver for Bareon [1].

Thanks,

[0] http://bareon-allocator.readthedocs.org/en/latest/architecture.html
[1] https://wiki.openstack.org/wiki/Bareon
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] github can't sync with git because of big uploading

2015-12-30 Thread Evgeniy L

Hi,

You should probably talk to infra team on this issue #openstack-infra
channel.
I see several ways what can be done here:
1. recreate the repo
2. create a repo with different name

Also there is possibility to remove from the commit those files and push
using --force,
but it's a very bad practise and I don't think that infra allows to do that.

Thanks,

On Wed, Dec 30, 2015 at 5:34 AM, wuwenbin  wrote:

> Hi all:
>
>  Repo of fuel-plugin-onos has something wrong because of uploading
> big file. Though codes are reverted while history still contains the pack
> which results in big downloading and unsync with github.
>
>  I really want to solve this problem and please forgive my own
> decision for a new commit of new onosfw because I don’t want this impacting
> the project. I have to admit that I am really bad at management of commit
> and merge. So I invite fuel ptl as the manager of new repo to avoid such
> things.
>
>  Does anyone can help me solve this as soon as possible?
>
>Thanks
>
> Bests
>
> Cathy
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Bareon] Fuel & Bareon integration (fuel modularisation)

2015-12-29 Thread Evgeniy L

Hi Aleksey,

Since Fuel's FF is on 2nd of March, I don't think that full switch from
fuel-agent + nailgun volume manager to Bareon will happen.
Also Bareon will have separate from Fuel release cycle, so you will have
to land the feature first into Bareon and then to Fuel.

Could you please provide a bit more information on what features are
needed in the agent for 9.0 Fuel release?

Thanks,

On Tue, Dec 29, 2015 at 1:38 PM, Aleksey Zvyagintsev <
azvyagint...@mirantis.com> wrote:

>
> Guys, do you have any plans to achieve integration for fuel9\fuel-next
> ?(and final switching from fuel-agent to BareOn will be...?)
> We are going to implement some new features for fuel - and the main
> question should we push it into fuel-agent or in newer BareOn?
>
>
> On Mon, Dec 28, 2015 at 11:56 PM, Tomasz Napierala <
> tnapier...@mirantis.com> wrote:
>
>> I agree with Evgeny: from work organization it would more optimal to have
>> 2 repos. API and system facing programming are completely different
>> domains, requiring different skill sets. In my opinion separation would
>> lower the entry barriers.
>>
>> Regards,
>>
>> > On 17 Dec 2015, at 15:53, Evgeniy L <e...@mirantis.com> wrote:
>> >
>> > Hi Igor,
>> >
>> > Bareon by itself doesn't have any REST interface, Bareon is basically
>> fuel_agent,
>> > which is framework + CLI wrapper to use it as an agent.
>> > In order to store and edit required entities in the database we need
>> some wrapper,
>> > which adds this functionality. This simple wrapper will be implemented
>> in Bareon-API.
>> > User should be able to use Bareon without any additional API/Database
>> if she/he
>> > wants to do some basic stuff without need to store the configuration,
>> which is not
>> > Fuel use case.
>> > If the question was specifically about Bareon-API in separate repo,
>> there is no
>> > reason to store it in a single repo, since we may have separate teams
>> working
>> > on those sub-projects and those solve a bit different problems, user
>> facing api
>> > vs low level tools.
>> >
>> > Thanks,
>> >
>> > On Thu, Dec 17, 2015 at 5:33 PM, Igor Kalnitsky <
>> ikalnit...@mirantis.com> wrote:
>> > > create Bareon-API repository, and start production ready
>> implementation
>> >
>> > For what reason do we need a separate repo? I thought API will be a
>> > part of bareon repo. Or bareon is just a provisioning agent, which
>> > will be driven by bareon-api?
>> >
>> > On Thu, Dec 17, 2015 at 12:29 PM, Evgeniy L <e...@mirantis.com> wrote:
>> > > Hi,
>> > >
>> > > Some time ago, we’ve started a discussion [0] about Fuel
>> modularisation
>> > > activity.
>> > > Due to unexpected circumstances POC has been delayed.
>> > >
>> > > Regarding to partitioning/provisioning system, we have POC with a
>> demo [1]
>> > > (thanks to Sylwester), which shows how the integration of Fuel and
>> Bareon
>> > > [2] can
>> > > be done.
>> > >
>> > > To summarise the implementation:
>> > > * we have a simple implementation of Bareon-API [3], which stores
>> > > partitioning
>> > >   related data and allows to edit it
>> > > * for Nailgun new extension has been implemented [4], which uses
>> Bareon-API
>> > >   to store partitioning information, so we will be able to easily
>> switch
>> > > between
>> > >   classic volume_manager implementation and Bareon-API extension
>> > > * when provisioning gets started, extensions retrieves the data from
>> > > Bareon-API
>> > >
>> > > Next steps:
>> > > * create Bareon-API repository, and start production ready
>> implementation
>> > > * create a spec for Fuel project
>> > > * create a spec for Bareon project
>> > >
>> > > If you have any questions don’t hesitate to ask them in this thread,
>> also
>> > > you can
>> > > find us on #openstack-bareon channel.
>> > >
>> > > Thanks!
>> > >
>> > > [0]
>> > >
>> http://lists.openstack.org/pipermail/openstack-dev/2015-October/077025.html
>> > > [1] https://www.youtube.com/watch?v=GTJM8i7DL0w
>> > > [2]
>> > >
>> http://lists.openstack.org/pipermail/openstack-dev/2015-December/082397.html
>> > > [3] https://github.com/Miranti

Re: [openstack-dev] [Fuel] Removal of support for nova-network

2015-12-22 Thread Evgeniy L

Hi,

We mustn't touch Nailgun's logic, otherwise after upgrade user won't be able
to manage her/his old nova Cluster. So lets just remove it from UI.
Also as far as I know we should provide a way to manage old clusters not
for a release, but for a couple of years.

Thanks,

On Tue, Dec 22, 2015 at 10:40 AM, Igor Kalnitsky 
wrote:

> I don't think it's a good idea to drop support of 7.0 nova-network
> setup in 8.0. We should keep compatibility for at least one release.
>
> On Tue, Dec 22, 2015 at 9:15 AM, Aleksey Kasatkin
>  wrote:
> > Sergii,
> >
> > We could remove it completely from nailgun if support for 7.0 and
> earlier is
> > not required.
> >
> >
> > Aleksey Kasatkin
> >
> >
> > On Tue, Dec 22, 2015 at 3:27 AM, Sergii Golovatiuk
> >  wrote:
> >>
> >> Hi,
> >>
> >> Finally we can deprecate nova-network ...
> >> We should remove it from UI, nailgun logic and tests to have less
> >> technical debt.
> >>
> >> --
> >> Best regards,
> >> Sergii Golovatiuk,
> >> Skype #golserge
> >> IRC #holser
> >>
> >> On Mon, Dec 21, 2015 at 5:01 PM, Sheena Gregson 
> >> wrote:
> >>>
> >>> Hey guys –
> >>>
> >>>
> >>>
> >>> I know this has been a topic of a lot of discussion – Adrian informed
> me
> >>> on Friday that QA has confirmed the multi-hypervisor use case has been
> >>> tested successfully without nova-network, so we can finally deprecate
> it!
> >>>
> >>>
> >>>
> >>> Users who want to deploy multiple hypervisors will need to use the Fuel
> >>> DVS plugin (Neutron ML2 driver) to support their vCenter computes and
> the
> >>> KVM/QEMU computes can use Neutron + GRE/VXLAN.
> >>>
> >>>
> >>>
> >>> I’ve created a kind of “cover all the things” bug here:
> >>> https://bugs.launchpad.net/fuel/+bug/1528407.  Given the state of
> >>> nova-network right now in Fuel, I have marked it as Critical.
> >>>
> >>>
> >>>
> >>> Let’s start the conversation on here and make sure all the bases are
> >>> covered – if additional bugs need to be logged or there’s
> administrative
> >>> overhead, let me know and I’ll be happy to help out!
> >>>
> >>>
> >>>
> >>> Sheena Gregson | Sr. Product Manager | Mirantis
> >>>
> >>> p: +1 650 646 3302 | e: sgreg...@mirantis.com
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> __
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>
> >>
> >>
> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Solar] SolarDB/ConfigDB place in Fuel

2015-12-21 Thread Evgeniy L

Hi Oleg,

I understand the concern, but in case of integration specifically with
Solar,
I don't see any reasons to add ConfigDB, because Solar by itself is a
ConfigDB.
At the same time I would agree that there might be a case, when user uses
Zookeeper/Puppet Master/CMDB as a data store, in this case we should store
the data directly in those services, without keeping them in yet another
storage.

So the flow will look like this:
Components ->
Data get polled by data processors ->
| Solar data processor puts the data into Solar in its format
| Zookeeper data processor puts the data into Zookeeper in its format
| Custom CMDB data processor puts the data into CMDB in its own format

Thanks,

On Fri, Dec 18, 2015 at 7:00 PM, Oleg Gelbukh <ogelb...@mirantis.com> wrote:

> Hi,
>
> The idea behind configdb is that it is independent component that defines
> data flows between other components of the system. It has no knowledge
> about those components or specifics of their data. Data formats are defined
> by components themselves via schemas/templates and can be changed at any
> time (i.e. don't require code changes).
>
> Important 'pro' having ConfigDB separate from Solar is that it will
> simplify transition from current Fuel architecture by breaking it into more
> definite stages and reducing the number of components Solar have to be
> integrated with.
>
> --
> Best regards,
> Oleg Gelbukh
>
> On Wed, Dec 16, 2015 at 4:38 PM, Evgeniy L <e...@mirantis.com> wrote:
>
>> Hi Dmitry,
>>
>> I also don't think that we should duplicate the data in configdb,
>> because in this case there will be +2 additional interfaces which
>> will require to covert the data into configdb and after that from
>> configdb to Solar, which seems redundant overhead.
>>
>> But we should be able to put the data directly to user's
>> CMDB/ZooKeeper/Puppet Master/etc.
>>
>> Thanks,
>>
>> On Wed, Dec 16, 2015 at 2:03 AM, Dmitriy Shulyak <dshul...@mirantis.com>
>> wrote:
>>
>>> Hello folks,
>>>
>>> This topic is about configuration storage which will connect data
>>> sources (nailgun/bareon/others) and orchestration. And right now we are
>>> developing two projects that will overlap a bit.
>>>
>>> I understand there is not enough context to dive into this thread right
>>> away, but i will appreciate if those people, who participated in design,
>>> will add their opinions/clarifications on this matter.
>>>
>>> Main disagreements
>>> ---
>>> 1. configdb should be passive, writing to configdb is someone else
>>> responsibility
>>> + simpler implementation, easier to use
>>> - we will need another component that will do writing, or split this
>>> responsibility somehow
>>>
>>> 2. can be used without other solar components
>>> + clear inteface between solar components and storage layer
>>> - additional work required to design/refactor communication layer
>>> between modules in solar
>>> - some data will be duplicated between solar orchestrator layer and
>>> configdb
>>>
>>> 3. templates for output
>>> technical detail, can be added on top of solardb if required
>>>
>>> Similar functionality
>>> --
>>> 1. Hierachical storage
>>> 2. Versioning of changes
>>> 3. Possibility to overwrite config values
>>> 4. Schema for inputs
>>>
>>> Overall it seems that we share same goals for both services,
>>> the difference lies in organizational and technical implementation
>>> details.
>>>
>>> Possible solutions
>>> 
>>> 1. develop configdb and solar with duplicated functionality
>>> - at least 2 additional components will be added to the picture,
>>> one is configdb, another one will need to sync data between configdb and
>>> solar
>>> - in some cases data in solar and configdb will be 100% duplicated
>>> - different teams will work on same functionality
>>> - integration of additional component for fuel will require integration
>>> with
>>> configdb and with solar
>>> + configdb will be independent from solar orchestration/other components
>>>
>>> 2. make service out of solardb, allign with configdb use cases
>>> + solardb will be independent from solar orchestration/other solar
>>> components
>>> + integration of fuel component will be easier than in 1st version
>>> + clarity about components responsibility and new ar

Re: [openstack-dev] [Fuel][Solar] SolarDB/ConfigDB place in Fuel

2015-12-21 Thread Evgeniy L

Comments inline.

On Mon, Dec 21, 2015 at 1:42 PM, Oleg Gelbukh <ogelb...@mirantis.com> wrote:

> The problem with this approach is that we can't manage the interfaces
> between components without changing the code of 2+ components (i.e. one
> that provides the data and all that consume it).
>

I'm not sure if understand you correctly, none of these approaches
require to change the components, we will have change a single
place, which is specific data processor.


>
> I also don't like polling model for data processors. In my understanding,
> components should push their changes through the pipeline. Although this is
> pure implementation detail and is not really important ATM.
>

We don't have any other choice, we don't control components, configuration
components are not going to implement Fuel specific logic, so Fuel has
to pool the data.


>
> The point is that for Solar integration, we still need integration points,
> and the less of them we have, the more simple the transition is going to
> be..
>

As described above, there will be a single integration point, data
processor.

>
> --
> Best regards,
> Oleg Gelbukh
>
> On Mon, Dec 21, 2015 at 11:32 AM, Evgeniy L <e...@mirantis.com> wrote:
>
>> Hi Oleg,
>>
>> I understand the concern, but in case of integration specifically with
>> Solar,
>> I don't see any reasons to add ConfigDB, because Solar by itself is a
>> ConfigDB.
>> At the same time I would agree that there might be a case, when user uses
>> Zookeeper/Puppet Master/CMDB as a data store, in this case we should store
>> the data directly in those services, without keeping them in yet another
>> storage.
>>
>> So the flow will look like this:
>> Components ->
>> Data get polled by data processors ->
>> | Solar data processor puts the data into Solar in its format
>> | Zookeeper data processor puts the data into Zookeeper in its format
>> | Custom CMDB data processor puts the data into CMDB in its own format
>>
>> Thanks,
>>
>> On Fri, Dec 18, 2015 at 7:00 PM, Oleg Gelbukh <ogelb...@mirantis.com>
>> wrote:
>>
>>> Hi,
>>>
>>> The idea behind configdb is that it is independent component that
>>> defines data flows between other components of the system. It has no
>>> knowledge about those components or specifics of their data. Data formats
>>> are defined by components themselves via schemas/templates and can be
>>> changed at any time (i.e. don't require code changes).
>>>
>>> Important 'pro' having ConfigDB separate from Solar is that it will
>>> simplify transition from current Fuel architecture by breaking it into more
>>> definite stages and reducing the number of components Solar have to be
>>> integrated with.
>>>
>>> --
>>> Best regards,
>>> Oleg Gelbukh
>>>
>>> On Wed, Dec 16, 2015 at 4:38 PM, Evgeniy L <e...@mirantis.com> wrote:
>>>
>>>> Hi Dmitry,
>>>>
>>>> I also don't think that we should duplicate the data in configdb,
>>>> because in this case there will be +2 additional interfaces which
>>>> will require to covert the data into configdb and after that from
>>>> configdb to Solar, which seems redundant overhead.
>>>>
>>>> But we should be able to put the data directly to user's
>>>> CMDB/ZooKeeper/Puppet Master/etc.
>>>>
>>>> Thanks,
>>>>
>>>> On Wed, Dec 16, 2015 at 2:03 AM, Dmitriy Shulyak <dshul...@mirantis.com
>>>> > wrote:
>>>>
>>>>> Hello folks,
>>>>>
>>>>> This topic is about configuration storage which will connect data
>>>>> sources (nailgun/bareon/others) and orchestration. And right now we are
>>>>> developing two projects that will overlap a bit.
>>>>>
>>>>> I understand there is not enough context to dive into this thread
>>>>> right away, but i will appreciate if those people, who participated in
>>>>> design, will add their opinions/clarifications on this matter.
>>>>>
>>>>> Main disagreements
>>>>> ---
>>>>> 1. configdb should be passive, writing to configdb is someone else
>>>>> responsibility
>>>>> + simpler implementation, easier to use
>>>>> - we will need another component that will do writing, or split this
>>>>> responsibility somehow
>>>>>
>>>>> 2. can be used without o

Re: [openstack-dev] [Fuel] PostgreSQL 9.3 and JSON operations

Hi Andrew,

It doesn't look fair at all to say that we use Postgres specific feature
for no reasons
or as you said "just because we want".
For example we used Arrays which fits pretty well for our roles usage,
which improved
readability and performance.
Or try to fit into relational system something like that [1], I don't think
that we will get
a good result.

P.S. sending a link to a holywar topic (schema vs schemaless), won't help
to solve our
specific problem with Postgres downgrading vs keeping old (new) version.

[1]
https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/fixtures/openstack.yaml


On Tue, Dec 15, 2015 at 10:53 PM, Andrew Maksimov 
wrote:

> +1 to Igor suggestion to downgrade Postgres to 9.2. Our users don't work
> directly with Postgres, so there is no any deprecation of Fuel features.
> Maintaining our own custom Postgres package just because we want "JSON
> column" is not a rational decision. Come on, fuel is not a billing system
> with thousands tables and special requirements to database. At least, we
> should try to keep it simple and avoid unnecessary complication.
>
> PS
>  BTW, some people suggest to avoid using  json columns, read [1]
> PostgreSQL anti-patterns: unnecessary json columns.
>
> [1] -
> http://blog.2ndquadrant.com/postgresql-anti-patterns-unnecessary-jsonhstore-dynamic-columns/
>
> Regards,
> Andrey Maximov
> Fuel Project Manager
>
>
> On Tue, Dec 15, 2015 at 9:34 PM, Vladimir Kuklin 
> wrote:
>
>> Folks
>>
>> Let me add my 2c here.
>>
>> I am for using Postgres 9.3. Here is an additional argument to the ones
>> provided by Artem, Aleksandra and others.
>>
>> Fuel is being sometimes highly customized by our users for their specific
>> needs. It has been Postgres 9.3 for a while and they might have as well
>> gotten used to it and assumed by default that this would not change. So
>> some of their respective features they are developing for their own sake
>> may depend on Postgres 9.3 and we will never be able to tell the fraction
>> of such use cases. Moreover, downgrading DBMS version of Fuel should be
>> inevitably considered as a 'deprecation' of some features our software
>> suite is providing to our users. This actually means that we MUST provide
>> our users with a warning and deprecation period to allow them to adjust to
>> these changes. Obviously, accidental change of Postgres version does not
>> follow such a policy in any way. So I see no other ways except for getting
>> back to Postgres 9.3.
>>
>>
>> On Tue, Dec 15, 2015 at 7:39 PM, Igor Kalnitsky 
>> wrote:
>>
>>> Hey Mike,
>>>
>>> Thanks for your input.
>>>
>>> > actually not.  if you replace your ARRAY columns with JSON entirely,
>>>
>>> It still needs to fix the code, i.e. change ARRAY-specific queries
>>> with JSON ones around the code. ;)
>>>
>>> > there's already a mostly finished PR for SQLAlchemy support in the
>>> queue.
>>>
>>> Does it mean SQLAlchemy will have one unified interface to make JSON
>>> queries? So we can use different backends if necessary?
>>>
>>> Thanks,
>>> - Igor
>>>
>>> On Tue, Dec 15, 2015 at 5:06 PM, Mike Bayer  wrote:
>>> >
>>> >
>>> > On 12/15/2015 07:20 AM, Igor Kalnitsky wrote:
>>> >> Hey Julien,
>>> >>
>>> >>>
>>> https://blueprints.launchpad.net/fuel/+spec/openstack-ha-fuel-postgresql
>>> >>
>>> >> I believe this blueprint is about DB for OpenStack cloud (we use
>>> >> Galera now), while here we're talking about DB backend for Fuel
>>> >> itself. Fuel has a separate node (so called Fuel Master) and we use
>>> >> PostgreSQL now.
>>> >>
>>> >>> does that mean Fuel is only going to be able to run with PostgreSQL?
>>> >>
>>> >> Unfortunately we already tied up to PostgreSQL. For instance, we use
>>> >> PostgreSQL's ARRAY column type. Introducing JSON column is one more
>>> >> way to tighten knots harder.
>>> >
>>> > actually not.  if you replace your ARRAY columns with JSON entirely,
>>> > MySQL has JSON as well now:
>>> > https://dev.mysql.com/doc/refman/5.7/en/json.html
>>> >
>>> > there's already a mostly finished PR for SQLAlchemy support in the
>>> queue.
>>> >
>>> >
>>> >
>>> >>
>>> >> - Igor
>>> >>
>>> >> On Tue, Dec 15, 2015 at 12:28 PM, Julien Danjou 
>>> wrote:
>>> >>> On Mon, Dec 14 2015, Igor Kalnitsky wrote:
>>> >>>
>>>  The things I want to notice are:
>>> 
>>>  * Currently we aren't tied up to PostgreSQL 9.3.
>>>  * There's a patch [2] that ties Fuel up to PostgreSQL 9.3+ by using
>>> a
>>>  set of JSON operations.
>>> >>>
>>> >>> I'm curious and have just a small side question: does that mean Fuel
>>> is
>>> >>> only going to be able to run with PostgreSQL?
>>> >>>
>>> >>> I also see
>>> >>>
>>> https://blueprints.launchpad.net/fuel/+spec/openstack-ha-fuel-postgresql
>>> ,
>>> >>> maybe it's related?
>>> >>>
>>> >>> Thanks!
>>> >>>
>>> >>> --
>>> >>> Julien Danjou
>>> >>> // Free Software hacker
>>> >>> //

Re: [openstack-dev] [Fuel] Proposal to Delay Docker Removal From Fuel Master Node

Hi Oleg,

With the same degree of confidence we can say that anything we have in the
beginning of
the release cycle is not urgent enough. We pushed early branching
specifically for
such big changes as Docker removal/Changing repos structures and merging
invasive patches
for new release features.

Vladimir Kuklin,

I'm not sure what do you mean by "fixing 2 different environments"? With
environment without
containers it will simplify debugging process.

Thanks,

On Wed, Dec 16, 2015 at 10:12 PM, Oleg Gelbukh <ogelb...@mirantis.com>
wrote:

> Hi
>
> Although I agree that it should be done, the removal of Docker doesn't
> seem an urgent feature to me. It is not blocking anything besides moving to
> full package-based deployment of Fuel, as far as I understand. So it could
> be easily delayed for one milestone, especially if it is already almost
> done and submitted for review, so it could be merged fast before any other
> significant changes land in 'master' after it is open.
>
> --
> Best regards,
> Oleg Gelbukh
>
> On Wed, Dec 16, 2015 at 8:56 PM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com> wrote:
>
>> Vladimir,
>>
>> I have other activities planned for the time immediately after SCF
>> (separating UI from fuel-web, maybe it is even more invasive :-)) and it is
>> not a big deal to postpone this feature or another. I am against the
>> approach itself of postponing something because it is too invasive. If we
>> create stable branch master becomes open. That was our primary intention to
>> open master earlier than later when we decided to move stable branch
>> creation.
>>
>>
>>
>>
>> Vladimir Kozhukalov
>>
>> On Wed, Dec 16, 2015 at 8:28 PM, Vladimir Kuklin <vkuk...@mirantis.com>
>> wrote:
>>
>>> Vladimir
>>>
>>> I am pretty much for removing docker, but I do not think that we should
>>> startle our developers/QA folks with additional efforts on fixing 2
>>> different environments. Let's just think from the point of development
>>> velocity here and at delay such changes for at least after NY. Because if
>>> we do it immediately after SCF there will be a whole bunch of holidays and
>>> Russian holidays are Jan 1st-10th and you (who is the SME for docker
>>> removal) will be offline. Do you really want to fix things instead of
>>> enjoying holidays?
>>>
>>> On Wed, Dec 16, 2015 at 4:09 PM, Evgeniy L <e...@mirantis.com> wrote:
>>>
>>>> +1 to Vladimir Kozhukalov,
>>>>
>>>> Entire point of moving branches creation to SCF was to perform such
>>>> changes as
>>>> early as possible in the release, I see no reasons to wait for HCF.
>>>>
>>>> Thanks,
>>>>
>>>> On Wed, Dec 16, 2015 at 10:19 AM, Vladimir Kozhukalov <
>>>> vkozhuka...@mirantis.com> wrote:
>>>>
>>>>> -1
>>>>>
>>>>> We already discussed this and we have made a decision to move stable
>>>>> branch creation from HCF to SCF. There were reasons for this. We agreed
>>>>> that once stable branch is created, master becomes open for new features.
>>>>> Let's avoid discussing this again.
>>>>>
>>>>> Vladimir Kozhukalov
>>>>>
>>>>> On Wed, Dec 16, 2015 at 9:55 AM, Bulat Gaifullin <
>>>>> bgaiful...@mirantis.com> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Regards,
>>>>>> Bulat Gaifullin
>>>>>> Mirantis Inc.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 15 Dec 2015, at 22:19, Andrew Maksimov <amaksi...@mirantis.com>
>>>>>> wrote:
>>>>>>
>>>>>> +1
>>>>>>
>>>>>> Regards,
>>>>>> Andrey Maximov
>>>>>> Fuel Project Manager
>>>>>>
>>>>>> On Tue, Dec 15, 2015 at 9:41 PM, Vladimir Kuklin <
>>>>>> vkuk...@mirantis.com> wrote:
>>>>>>
>>>>>>> Folks
>>>>>>>
>>>>>>> This email is a proposal to push Docker containers removal from the
>>>>>>> master node to the date beyond 8.0 HCF.
>>>>>>>
>>>>>>> Here is why I propose to do so.
>>>>>>>
>>>>>>> Removal of Docker is a rather invasive change and may intr

[openstack-dev] [Fuel][Bareon] Fuel & Bareon integration (fuel modularisation)

Hi,

Some time ago, we’ve started a discussion [0] about Fuel modularisation
activity.
Due to unexpected circumstances POC has been delayed.

Regarding to partitioning/provisioning system, we have POC with a demo [1]
(thanks to Sylwester), which shows how the integration of Fuel and Bareon
[2] can
be done.

To summarise the implementation:
* we have a simple implementation of Bareon-API [3], which stores
partitioning
  related data and allows to edit it
* for Nailgun new extension has been implemented [4], which uses Bareon-API
  to store partitioning information, so we will be able to easily switch
between
  classic volume_manager implementation and Bareon-API extension
* when provisioning gets started, extensions retrieves the data from
Bareon-API

Next steps:
* create Bareon-API repository, and start production ready implementation
* create a spec for Fuel project
* create a spec for Bareon project

If you have any questions don’t hesitate to ask them in this thread, also
you can
find us on #openstack-bareon channel.

Thanks!

[0]
http://lists.openstack.org/pipermail/openstack-dev/2015-October/077025.html
[1] https://www.youtube.com/watch?v=GTJM8i7DL0w
[2]
http://lists.openstack.org/pipermail/openstack-dev/2015-December/082397.html
[3] https://github.com/Mirantis/bareon-api
[4] https://review.openstack.org/#/c/250864/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] PostgreSQL 9.3 and JSON operations

Hi,

Since older Postgres doesn't introduce bugs and it won't harm new features,
I would vote for downgrade to 9.2

The reasons are:
1. not to support own package for Centos (as far as I know 9.3 for Ubuntu
is already there)
2. should Fuel some day be a part of upstream Centos? If yes, or there is
even small probability that
it's going to be, we should be as much as possible compatible with
upstream repo. If we don't
consider such possibility, it doesn't really matter, because user will
have to connect external
repo anyway.

Since we already use Postgres specific features, we should spawn a separate
thread, if
we should or shouldn't continue doing that, and if there is a real need to
support mysql
for example.

Thanks,

On Wed, Dec 16, 2015 at 3:58 PM, Igor Kalnitsky 
wrote:

> > From what I understand, we are using 9.2 since the CentOS 7 switch. Can
> > anyone point me to a bug caused by that?
>
> AFAIK, there's no such bugs. Some folks have just *concerns*. Anyway,
> it's up to packaging team to decide whether to package or not.
>
> From Nailgun POV, I'd like to see classical RDBMS schemas as much as
> possible, and do not rely on database backend and its version.
>
> On Wed, Dec 16, 2015 at 11:30 AM, Bartłomiej Piotrowski
>  wrote:
> > On 2015-12-16 10:14, Bartłomiej Piotrowski wrote:
> >> On 2015-12-16 08:23, Mike Scherbakov wrote:
> >>> We could consider downgrading in Fuel 9.0, but I'd very carefully
> >>> consider that. As Vladimir Kuklin said, there are may be other users
> who
> >>> already rely on 9.3 for some of their enhancements.
> >>
> >> That will be way too late for that, as it will make upgrade procedure
> >> more complicated. Given no clear upgrade path from 7.0 to 8.0, it sounds
> >> like perfect opportunity to use what is provided by base distribution.
> >> Are there actual users facilitating 9.3 features or is it some kind of
> >> Invisible Pink Unicorn?
> >>
> >> Bartłomiej
> >>
> >
> > I also want to remind that we are striving for possibility to let users
> > do 'yum install fuel' (or apt) to make the magic happen. There is not
> > much magic in requiring potential users to install specific PostgreSQL
> > version because someone said so. It's either supporting the lowest
> > version available (CentOS 7 – 9.2, Ubuntu 14.04 – 9.3, Debian Jessie –
> > 9.4, openSUSE Leap – 9.4) or "ohai add this repo with our manually
> > imported and rebuilt EPEL package".
> >
> > From what I understand, we are using 9.2 since the CentOS 7 switch. Can
> > anyone point me to a bug caused by that?
> >
> > BP
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Bareon] Fuel & Bareon integration (fuel modularisation)

Hi Igor,

Bareon by itself doesn't have any REST interface, Bareon is basically
fuel_agent,
which is framework + CLI wrapper to use it as an agent.
In order to store and edit required entities in the database we need some
wrapper,
which adds this functionality. This simple wrapper will be implemented in
Bareon-API.
User should be able to use Bareon without any additional API/Database if
she/he
wants to do some basic stuff without need to store the configuration, which
is not
Fuel use case.
If the question was specifically about Bareon-API in separate repo, there
is no
reason to store it in a single repo, since we may have separate teams
working
on those sub-projects and those solve a bit different problems, user facing
api
vs low level tools.

Thanks,

On Thu, Dec 17, 2015 at 5:33 PM, Igor Kalnitsky <ikalnit...@mirantis.com>
wrote:

> > create Bareon-API repository, and start production ready implementation
>
> For what reason do we need a separate repo? I thought API will be a
> part of bareon repo. Or bareon is just a provisioning agent, which
> will be driven by bareon-api?
>
> On Thu, Dec 17, 2015 at 12:29 PM, Evgeniy L <e...@mirantis.com> wrote:
> > Hi,
> >
> > Some time ago, we’ve started a discussion [0] about Fuel modularisation
> > activity.
> > Due to unexpected circumstances POC has been delayed.
> >
> > Regarding to partitioning/provisioning system, we have POC with a demo
> [1]
> > (thanks to Sylwester), which shows how the integration of Fuel and Bareon
> > [2] can
> > be done.
> >
> > To summarise the implementation:
> > * we have a simple implementation of Bareon-API [3], which stores
> > partitioning
> >   related data and allows to edit it
> > * for Nailgun new extension has been implemented [4], which uses
> Bareon-API
> >   to store partitioning information, so we will be able to easily switch
> > between
> >   classic volume_manager implementation and Bareon-API extension
> > * when provisioning gets started, extensions retrieves the data from
> > Bareon-API
> >
> > Next steps:
> > * create Bareon-API repository, and start production ready implementation
> > * create a spec for Fuel project
> > * create a spec for Bareon project
> >
> > If you have any questions don’t hesitate to ask them in this thread, also
> > you can
> > find us on #openstack-bareon channel.
> >
> > Thanks!
> >
> > [0]
> >
> http://lists.openstack.org/pipermail/openstack-dev/2015-October/077025.html
> > [1] https://www.youtube.com/watch?v=GTJM8i7DL0w
> > [2]
> >
> http://lists.openstack.org/pipermail/openstack-dev/2015-December/082397.html
> > [3] https://github.com/Mirantis/bareon-api
> > [4] https://review.openstack.org/#/c/250864/
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [All][Bareon][Fuel][Ironic] New project: Bareon (provisioning system)

2015-12-16 Thread Evgeniy L

We are happy to introduce Bareon [0], which is a fork of fuel_agent [1]
project which have been developed under Fuel’s umbrella.

Bareon provides flexible and data driven interface to perform actions
which are related to operating system installation. In contrary to standard
kickstart and preseed ways of installation, Bareon provides a uniform
interface for system configuration. Higher level of flexibility is open via
this interface for other tools and products, such as Ironic [2] or Fuel.
It is to be a tool similar to old plain OS installation tools like anaconda
or debian-installer but drastically more convenient.

To summarise what Bareon is:
* flexible partitioning system
* image building functionality, mostly non-cloud use-cases that assume
  advanced partitioning, OS upgrading, in-place image building
* image/repo based provisioning

We decided to continue developing it as separate from Fuel project for
several reasons:
* it solves a set of problems which are not Fuel specific, with “fuel”
prefix
  in the name people may think that it’s intended to be used only with Fuel
* it's easier to contribute to a small standalone project than to find your
way
  around all the components of a large project such as Fuel
* to have separate roadmap [3] with its own release cycle and team, we
  really want to have small releases more often, it’ll be easier to do for a
  small project

fuel_agent repository and the project will exist for some time not to
disrupt
Fuel’s roadmap and its release cycle, after we finish full renaming and
required
refactoring, Fuel will be switched to Bareon.

Currently we are working on cleaning repository to finish renaming [4].
In case if you have any questions join us on #openstack-bareon irc channel.
We are going to arrange Bareon meetings soon, stay tuned.

Thanks!

[0] https://wiki.openstack.org/wiki/Bareon
[1] https://github.com/openstack/fuel-agent
[2]
https://github.com/openstack/bareon/blob/master/contrib/ironic/ironic-fa-deploy/README.rst
[3] https://wiki.openstack.org/wiki/Bareon/Roadmap
[4] https://review.openstack.org/#/c/257836/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Solar] SolarDB/ConfigDB place in Fuel

2015-12-16 Thread Evgeniy L

Hi Dmitry,

I also don't think that we should duplicate the data in configdb,
because in this case there will be +2 additional interfaces which
will require to covert the data into configdb and after that from
configdb to Solar, which seems redundant overhead.

But we should be able to put the data directly to user's
CMDB/ZooKeeper/Puppet Master/etc.

Thanks,

On Wed, Dec 16, 2015 at 2:03 AM, Dmitriy Shulyak 
wrote:

> Hello folks,
>
> This topic is about configuration storage which will connect data sources
> (nailgun/bareon/others) and orchestration. And right now we are developing
> two projects that will overlap a bit.
>
> I understand there is not enough context to dive into this thread right
> away, but i will appreciate if those people, who participated in design,
> will add their opinions/clarifications on this matter.
>
> Main disagreements
> ---
> 1. configdb should be passive, writing to configdb is someone else
> responsibility
> + simpler implementation, easier to use
> - we will need another component that will do writing, or split this
> responsibility somehow
>
> 2. can be used without other solar components
> + clear inteface between solar components and storage layer
> - additional work required to design/refactor communication layer between
> modules in solar
> - some data will be duplicated between solar orchestrator layer and
> configdb
>
> 3. templates for output
> technical detail, can be added on top of solardb if required
>
> Similar functionality
> --
> 1. Hierachical storage
> 2. Versioning of changes
> 3. Possibility to overwrite config values
> 4. Schema for inputs
>
> Overall it seems that we share same goals for both services,
> the difference lies in organizational and technical implementation details.
>
> Possible solutions
> 
> 1. develop configdb and solar with duplicated functionality
> - at least 2 additional components will be added to the picture,
> one is configdb, another one will need to sync data between configdb and
> solar
> - in some cases data in solar and configdb will be 100% duplicated
> - different teams will work on same functionality
> - integration of additional component for fuel will require integration
> with
> configdb and with solar
> + configdb will be independent from solar orchestration/other components
>
> 2. make service out of solardb, allign with configdb use cases
> + solardb will be independent from solar orchestration/other solar
> components
> + integration of fuel component will be easier than in 1st version
> + clarity about components responsibility and new architecture
> - redesign/refactoring communication between components in solar
>
> 3. do not use configdb/no extraction of solardb
> - inproc communication, which can lead to coupled components (not the case
> currently)
> + faster implementation (no major changes required for integration with
> fuel)
> + clarity about components responsibility and new architecture
>
> Summary
> -
> For solar it makes no difference where data will come from: configdb or
> data sources, but in overall fuel architecture it will lead to significant
> complexity increase.
> It would be the best to follow 2nd path, because in long term we don't
> want tightly coupled components, but in nearest future we need to
> concentrate
> on:
> - integration with fuel
> - implementing policy engine
> - polishing solar components
> This is why i am not sure that we can spend time on 2nd path right now,
> or even before 9.0.
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Proposal to Delay Docker Removal From Fuel Master Node

2015-12-16 Thread Evgeniy L

+1 to Vladimir Kozhukalov,

Entire point of moving branches creation to SCF was to perform such changes
as
early as possible in the release, I see no reasons to wait for HCF.

Thanks,

On Wed, Dec 16, 2015 at 10:19 AM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> -1
>
> We already discussed this and we have made a decision to move stable
> branch creation from HCF to SCF. There were reasons for this. We agreed
> that once stable branch is created, master becomes open for new features.
> Let's avoid discussing this again.
>
> Vladimir Kozhukalov
>
> On Wed, Dec 16, 2015 at 9:55 AM, Bulat Gaifullin 
> wrote:
>
>> +1
>>
>> Regards,
>> Bulat Gaifullin
>> Mirantis Inc.
>>
>>
>>
>> On 15 Dec 2015, at 22:19, Andrew Maksimov  wrote:
>>
>> +1
>>
>> Regards,
>> Andrey Maximov
>> Fuel Project Manager
>>
>> On Tue, Dec 15, 2015 at 9:41 PM, Vladimir Kuklin 
>> wrote:
>>
>>> Folks
>>>
>>> This email is a proposal to push Docker containers removal from the
>>> master node to the date beyond 8.0 HCF.
>>>
>>> Here is why I propose to do so.
>>>
>>> Removal of Docker is a rather invasive change and may introduce a lot of
>>> regressions. It is well may affect how bugs are fixed - we might have 2
>>> ways of fixing them, while during SCF of 8.0 this may affect velocity of
>>> bug fixing as you need to fix bugs in master prior to fixing them in stable
>>> branches. This actually may significantly increase our bugfixing pace and
>>> put 8.0 GA release on risk.
>>>
>>>
>>>
>>> --
>>> Yours Faithfully,
>>> Vladimir Kuklin,
>>> Fuel Library Tech Lead,
>>> Mirantis, Inc.
>>> +7 (495) 640-49-04
>>> +7 (926) 702-39-68
>>> Skype kuklinvv
>>> 35bk3, Vorontsovskaya Str.
>>> Moscow, Russia,
>>> www.mirantis.com 
>>> www.mirantis.ru
>>> vkuk...@mirantis.com
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> 
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org
>> ?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Nominate Bulat Gaifulin for fuel-web & fuel-mirror cores

2015-12-15 Thread Evgeniy L

+1

On Tue, Dec 15, 2015 at 12:33 PM, Anastasia Urlapova  wrote:

> +1
>
> On Mon, Dec 14, 2015 at 3:20 PM, Roman Vyalov 
> wrote:
>
>> +1
>>
>> On Mon, Dec 14, 2015 at 3:05 PM, Aleksey Kasatkin > > wrote:
>>
>>> +1.
>>>
>>>
>>> Aleksey Kasatkin
>>>
>>>
>>> On Mon, Dec 14, 2015 at 12:49 PM, Vladimir Sharshov <
>>> vshars...@mirantis.com> wrote:
>>>
 Hi,

 +1 from me to Bulat.

 On Mon, Dec 14, 2015 at 1:03 PM, Igor Kalnitsky <
 ikalnit...@mirantis.com> wrote:

> Hi Fuelers,
>
> I'd like to nominate Bulat Gaifulin [1] for
>
> * fuel-web-core [2]
> * fuel-mirror-core [3]
>
> Bulat's doing a really good review with detailed feedback and he's a
> regular participant in IRC. He's co-author of packetary and
> fuel-mirror projects, and he made valuable contribution to fuel-web
> (e.g. task-based deployment engine).
>
> Fuel Cores, please reply back with +1/-1.
>
> - Igor
>
> [1] http://stackalytics.com/?module=fuel-web_id=bgaifullin
> [2] http://stackalytics.com/report/contribution/fuel-web/90
> [3] http://stackalytics.com/report/contribution/fuel-mirror/90
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Experimental Task Based Deployment Landed into Master

2015-12-14 Thread Evgeniy L

+1 It's really good job folks.

On Sat, Dec 12, 2015 at 2:25 AM, Vladimir Kuklin 
wrote:

> Fuelers
>
> I am thrilled to announce that task based deployment engine [0] has been
> just merged into Fuel master. We checked it against existing BVT test cases
> for regressions as well as against functional testing for several cases of
> deployment. All the OSTF and network verification tests have successfully
> passed.
>
> We will obviously need to polish it and fix bugs which will arise, but
> this is a gigantic step forward for our orchestration engine which should
> allow us to drastically increase our development velocity as well as end
> user experience.
>
> Thanks to all who participated in development testing and review:
>
> Dmitry Ilyin
> Vladimir Sharshov
> Bulat Gaifullin
> Alexey Shtokolov
> Igor Kalnitsky
> Evgeniy Li
> Sergii Golovatiuk
> Dmitry Shulyak
>
> and many-many others
>
> I am pretty confident that this will allow us to develop and test faster
> as well as introduce support of some of Life-Cycle Management scenarios in
> 8.0 release.
>
> Once again, thank you all, folks, for your dedicated work and efforts on
> making Fuel better.
>
> [0]
> https://blueprints.launchpad.net/fuel/+spec/task-based-deployment-astute
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com 
> www.mirantis.ru
> vkuk...@mirantis.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Dropping Python 2.6

2015-12-14 Thread Evgeniy L

Hi Roman,

We've discussed it [1], so +1

[1]
https://openstack.nimeyo.com/67521/openstack-dev-fuel-dropping-python2-6-compatibility

On Mon, Dec 14, 2015 at 5:05 PM, Roman Prykhodchenko  wrote:

> Fuelers,
>
> Since Mitaka OpenStack Infra has no resources to test python 2.6 support
> so the corresponding jobs are not running anymore. Since Fuel master node
> is on CentOS 7 now, let’s drop Python 2.6 support in Fuel.
>
>
> - romcheg
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Christmas Core Cleanup

2015-12-10 Thread Evgeniy L

Hi,

Thank you Igor for cleaning core related groups.

Also I would like to add that many of removed cores are still SME (subject
matter experts)
in specific areas, so they will continue reviewing related patches.

Thanks,

On Thu, Dec 10, 2015 at 2:42 PM, Igor Kalnitsky <ikalnit...@mirantis.com>
wrote:

> Hey folks,
>
> In an effort to do some housekeeping, I clean up the list of core
> reviewers in Fuel.
>
> According to Stackalytics the following cores show a low contribution rate:
>
> # fuel-web [1]
>
>     * Dmitry Shulyak
> * Evgeniy L
>
> # python-fuelclient [2]
>
> * Dmitry Pyzhov
> * Evgeniy L
>
> # shotgun [3]
>
> * Dmitry Shulyak
> * Evgeniy L
>
> # fuel-upgrade [4]
>
> * Aleksey Kasatkin
> * Vladimir Kozhukalov
>
> # fuel-main [5]
>
>     * Dmitry Pyzhov
> * Roman Vyalov
>
> # fuel-agent [6]
>
> * Aleksey Kasatkin
> * Evgeniy L
> * Igor Kalnitsky
>
> Also, I've removed Sebastian Kalinowski as he said he has no time to
> work on Fuel anymore.
>
> Once former cores show high level of contribution again, I'll gladly
> add them back.
>
> - Igor
>
> [1] http://stackalytics.com/report/contribution/fuel-web/90
> [2] http://stackalytics.com/report/contribution/python-fuelclient/90
> [3] http://stackalytics.com/report/contribution/shotgun/90
> [4] http://stackalytics.com/report/contribution/fuel-upgrade/90
> [5] http://stackalytics.com/report/contribution/fuel-main/90
> [6] http://stackalytics.com/report/contribution/fuel-agent/90
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Dropping python2.6 compatibility

2015-12-03 Thread Evgeniy L

Hi,

During Fuel master migration to CentOS7 there was found a problem that tests
get failed [0] for python2.6

As far as I can see it's a common practise to drop python2.6 compatibility
[1],
shouldn't we switch tests to work with python2.7 instead of python2.6?

It looks like fuelclient will be an exception, because clients continue
supporting
python2.6 [1]

Thanks,

[0] https://review.openstack.org/#/c/251120/
[1]
https://wiki.openstack.org/wiki/Python3#Python_2:_Python_2.6_support_dropped.2C_Python_2.7_only
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Patch size limit

2015-12-01 Thread Evgeniy L

Hi Maciej, thank you for bringing this up,

+1, but we should discuss the limit, personally for me it's ok to review
400loc patches,
if the patch covers only one bug-fix/feature implementation.

So if everybody is agree, we should:
1. update contribution guide
2. create a task for *non-voting* gate, which will -1 patch if it has loc
>= x (400?),
also as Neil mentioned, it shouldn't take into account files renaming.

Thanks,

On Tue, Dec 1, 2015 at 3:40 PM, Maciej Kwiek  wrote:

> Hi,
>
> I recently noticed the influx of big patches hitting Gerrit (especially in
> fuel-web, but I also heard that there was a couple of big ones in library).
> I think that patches that have 1000 LOC are simply too big to review
> thoroughly and reliably.
>
> I would argue that there should be a limit to patch size, either a soft
> one (i.e. written down in contributor guidelines) or a hard one (e.g.
> enforced by gate job).
>
> I think that we need a discussion whether we really need this limit, what
> should it be, and how to enforce it.
>
> I personally think that most patches that are over 400 LOC could be easily
> split into at least two smaller changes.
>
> Regarding the limit enforcement - I think we should go with the soft
> limit, with X LOC written as a guideline and giving the reviewers a
> possibility to give -1s to patches that are too big, but also giving the
> possibility to merge bigger changes if it's absolutely necessary (in really
> rare cases the changes simply cannot be split). We may mix in the hard
> limit for ridiculously large patches (twice the "soft limit" would be good
> in my opinion), so the gate would automatically reject such patches,
> forcing contributor to split his patch.
>
> Please share your thoughts on this.
>
> Regards,
> Maciej Kwiek
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Fuel Python Gerrit Dashboard

2015-11-30 Thread Evgeniy L

Thanks Igor, it's very helpful,

Also if you forget where to get the link, use the documentation [1].
I think something like that may also be useful for library and QA team.

Thanks,

[1] https://wiki.openstack.org/wiki/Fuel#Development_related_links


On Mon, Nov 30, 2015 at 3:29 PM, Igor Kalnitsky 
wrote:

> Hey Fuelers,
>
> As a part of improving review process, I've prepared a Gerrit
> Dashboard for Python projects [1]. First and foremost, I did it for
> myself (I believe it will help me to get attention to *ready-to-merge*
> patches), but I want to share the link with you.
>
> Feel free to use it!
>
> [1] https://goo.gl/aFHdeQ
>
> Thanks,
> Igor
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] [Fuel UI] Support of separate provisioning is blocked by backend issues

2015-11-23 Thread Evgeniy L

Hi,

I have several comments, just to make sure, that we are on the same page
here.
Current API calls for provisioning/deployment are used by developers and
fuel hackers,
and by design there was removed all validation. So shouldn't there be some
more
user friendly API calls which have validation? For example we don't run any
pre
deployment checks and network validation, you can even ask to deploy
offline nodes.
As result novice user can easily break her/his cluster.

Thanks,

On Fri, Nov 13, 2015 at 11:46 AM, Julia Aranovich 
wrote:

> Hi fuelers,
>
> Currently Fuel UI team is working on blueprint [1] to give Fuel UI user an
> ability to launch provisioning of environment nodes separately from
> deployment (without choosing particular nodes for now).
>
> In the process we were faced with the following issues. Some of them block
> the blueprint:
>
>- deployment constantly failed on environment with pre-provisioned
>nodes [2]
>- node pending_addition flag is reset to False for provisioned nodes
>[3]. This causes a lot of UX problems: provisioned node roles, disks,
>interfaces can not be reconfigured, node can not be deleted from
>environment, just can be marked as pending deletion (that requires
>environment deployment)
>- completed provisioning task has Null message. So, there is no to
>show the user after provisioning finished [4]
>- no notification comes on UI after provisioned finished [5]
>- fake provisioning task is also should be fixed: environment nodes
>stay in 'provisioning' status after provisioning finished [6]. This breaks
>fake Fuel UI workflow and brings difficulties in Fuel UI development.
>
> Could you please consider/fix the tickets and help to unblock the
> blueprint targeted for the current release.
>
> Also, you can check how provisioning works in Fuel UI on #547 custom 8.0
> ISO.
>
> Thank you!
> Julia
>
> [1]
> https://blueprints.launchpad.net/fuel/+spec/support-separate-provisioning-and-deployment-in-ui
> [2] https://bugs.launchpad.net/fuel/+bug/1515903
> [3] https://bugs.launchpad.net/fuel/+bug/1515898
> [4] https://bugs.launchpad.net/fuel/+bug/1515895
> [5] https://bugs.launchpad.net/fuel/+bug/1515891
> [6] https://bugs.launchpad.net/fuel/+bug/1515893
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Master node upgrade

2015-11-10 Thread Evgeniy L

Hi Vladimir,

Just to make sure that we are on the same page. We'll have to use upgrade
script anyway, since you will need to run database migration and register
new releases.

Thanks,

On Monday, 9 November 2015, Vladimir Kozhukalov <vkozhuka...@mirantis.com>
wrote:

> Looks like most people thing that building backup/re-install approach is
> more viable. So, we certainly need to invent completely new upgrade from
> and thus my suggestion is disable building/testing upgrade tarball right
> now, because anyway it makes no sense.
>
>
> Vladimir Kozhukalov
>
> On Fri, Nov 6, 2015 at 8:21 PM, Vladimir Kuklin <vkuk...@mirantis.com
> <javascript:_e(%7B%7D,'cvml','vkuk...@mirantis.com');>> wrote:
>
>> Just my 2 cents here - let's do docker backup and roll it up onto brand
>> new Fuel 8 node.
>>
>> On Fri, Nov 6, 2015 at 7:54 PM, Oleg Gelbukh <ogelb...@mirantis.com
>> <javascript:_e(%7B%7D,'cvml','ogelb...@mirantis.com');>> wrote:
>>
>>> Matt,
>>>
>>> You are talking about this part of Operations guide [1], or you mean
>>> something else?
>>>
>>> If yes, then we still need to extract data from backup containers. I'd
>>> prefer backup of DB in simple plain text file, since our DBs are not that
>>> big.
>>>
>>> [1]
>>> https://docs.mirantis.com/openstack/fuel/fuel-7.0/operations.html#howto-backup-and-restore-fuel-master
>>>
>>> --
>>> Best regards,
>>> Oleg Gelbukh
>>>
>>> On Fri, Nov 6, 2015 at 6:03 PM, Matthew Mosesohn <mmoses...@mirantis.com
>>> <javascript:_e(%7B%7D,'cvml','mmoses...@mirantis.com');>> wrote:
>>>
>>>> Oleg,
>>>>
>>>> All the volatile information, including a DB dump, are contained in the
>>>> small Fuel Master backup. There should be no information lost unless there
>>>> was manual customization done inside the containers (such as puppet
>>>> manifest changes). There shouldn't be a need to back up the entire
>>>> containers.
>>>>
>>>> The information we would lose would include the IP configuration
>>>> interfaces besides the one used for the Fuel PXE network and any custom
>>>> configuration done on the Fuel Master.
>>>>
>>>> I want #1 to work smoothly, but #2 should also be a safe route.
>>>>
>>>> On Fri, Nov 6, 2015 at 5:39 PM, Oleg Gelbukh <ogelb...@mirantis.com
>>>> <javascript:_e(%7B%7D,'cvml','ogelb...@mirantis.com');>> wrote:
>>>>
>>>>> Evgeniy,
>>>>>
>>>>> On Fri, Nov 6, 2015 at 3:27 PM, Evgeniy L <e...@mirantis.com
>>>>> <javascript:_e(%7B%7D,'cvml','e...@mirantis.com');>> wrote:
>>>>>
>>>>>> Also we should decide when to run containers
>>>>>> upgrade + host upgrade? Before or after new CentOS is installed?
>>>>>> Probably
>>>>>> it should be done before we run backup, in order to get the latest
>>>>>> scripts for
>>>>>> backup/restore actions.
>>>>>>
>>>>>
>>>>> We're working to determine if we need to backup/upgrade containers at
>>>>> all. My expectation is that we should be OK with just backup of DB, IP
>>>>> addresses settings from astute.yaml for the master node, and credentials
>>>>> from configuration files for the services.
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Oleg Gelbukh
>>>>>
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> On Fri, Nov 6, 2015 at 1:29 PM, Vladimir Kozhukalov <
>>>>>> vkozhuka...@mirantis.com
>>>>>> <javascript:_e(%7B%7D,'cvml','vkozhuka...@mirantis.com');>> wrote:
>>>>>>
>>>>>>> Dear colleagues,
>>>>>>>
>>>>>>> At the moment I'm working on deprecating Fuel upgrade tarball.
>>>>>>> Currently, it includes the following:
>>>>>>>
>>>>>>> * RPM repository (upstream + mos)
>>>>>>> * DEB repository (mos)
>>>>>>> * openstack.yaml
>>>>>>> * version.yaml
>>>>>>> * upgrade script itself (+ virtualenv)
>>>>>>>
>>>>>>> Apart from upgrading docker containers this upgrade script makes
>>>>>>> copies o

Re: [openstack-dev] [Fuel] shotgun code freeze

2015-11-10 Thread Evgeniy L

+1 to Dmitry, thanks for pushing this activity Vladimir.

On Friday, 6 November 2015, Dmitry Pyzhov  wrote:

> Great job! We are much closer to removal of fuel-web repo.
>
> On Tue, Oct 27, 2015 at 7:35 PM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com
> > wrote:
>
>> Dear colleagues,
>>
>> I am glad to announce that since now shotgun is a separate project.
>> fuel-web/shotgun directory has been deprecated. There is yet another patch
>> that has not been merged https://review.openstack.org/238525 (adds
>> .gitignore file to the new shotgun repo). Please review it.
>>
>> Shotgun
>>
>>- Launchpad bug https://bugs.launchpad.net/fuel/+bug/1506894
>>- project-config patch https://review.openstack.org/235355 (DONE)
>>- pypi (DONE)
>>- run_tests.sh https://review.openstack.org/235368 (DONE)
>>- rpm/deb specs https://review.openstack.org/#/c/235382/ (DONE)
>>- fuel-ci verification jobs https://review.fuel-infra.org/12872 (DONE)
>>- label jenkins slaves for verification (DONE)
>>- directory freeze (DONE)
>>- prepare upstream (DONE)
>>- waiting for project-config patch to be merged (DONE)
>>- .gitreview https://review.openstack.org/238476 (DONE)
>>- .gitignore https://review.openstack.org/238525 (ON REVIEW)
>>- custom jobs parameters https://review.fuel-infra.org/13209 (DONE)
>>- fix core group (DONE)
>>- fuel-main https://review.openstack.org/#/c/238953/ (DONE)
>>- packaging-ci  https://review.fuel-infra.org/13181 (DONE)
>>- MAINTAINERS https://review.openstack.org/239410 (DONE)
>>- deprecate shotgun directory https://review.openstack.org/239407
>>(DONE)
>>- fix verify-fuel-web-docs job (it installs shotgun for some reason)
>>https://review.fuel-infra.org/#/c/13194/ (DONE)
>>- remove old shotgun package (DONE)
>>
>>
>>
>> Vladimir Kozhukalov
>>
>> On Wed, Oct 21, 2015 at 2:46 PM, Vladimir Kozhukalov <
>> vkozhuka...@mirantis.com
>> > wrote:
>>
>>> Dear colleagues,
>>>
>>> As you might know I'm working on splitting multiproject fuel-web
>>> repository into several sub-projects. Shotgun is one of directories that
>>> are to be moved to a separate git project.
>>>
>>> Checklist for this to happen is as follows:
>>>
>>>- Launchpad bug https://bugs.launchpad.net/fuel/+bug/1506894
>>>- project-config patch  https://review.openstack.org/#/c/235355 (ON
>>>REVIEW)
>>>- pypi project
>>>https://pypi.python.org/pypi?%3Aaction=pkg_edit=Shotgun (DONE)
>>>- run_tests.sh  https://review.openstack.org/235368 (DONE)
>>>- rpm/deb specs  https://review.openstack.org/#/c/235382 (DONE)
>>>- fuel-ci verification jobs https://review.fuel-infra.org/#/c/12872/ (ON
>>>REVIEW)
>>>- label jenkins slaves for verification jobs (ci team)
>>>- directory freeze (WE ARE HERE)
>>>- prepare upstream (TODO)
>>>- waiting for project-config patch to be merged (ON REVIEW)
>>>- fuel-main patch (TODO)
>>>- packaging-ci patch (TODO)
>>>- deprecate fuel-web/shotgun directory (TODO)
>>>
>>> Now we are at the point where we need to freeze fuel-web/shotgun
>>> directory. So, I'd like to announce code freeze for this directory and all
>>> patches that make changes in the directory and are currently on review will
>>> need to be backported to the new git repository.
>>>
>>> Vladimir Kozhukalov
>>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins] Role for Fuel Master Node

2015-11-06 Thread Evgeniy L

Javeria,

In your case, I think it's easier to generate config on the target node,
using puppet for example, since the information which you may need
is placed in /etc/astute.yaml file. Also it may be a problem to retrieve
all required information about the cluster, since API is protected with
keystone authentication.

Thanks,

On Thu, Nov 5, 2015 at 5:35 PM, Javeria Khan  wrote:

> Hi Evgeniy,
>
>>
>> 1. what version of Fuel do you use?
>>
> Using 7.0
>
>
>> 2. could you please clarify what did you mean by "moving to
>> deployment_tasks.yaml"?
>>
> I tried changing my tasks.yaml to a deployment_tasks.yaml as the wiki
> suggests for 7.0. However I kept hitting issues.
>
>
>> 3. could you please describe your use-case a bit more? Why do you want to
>> run
>> tasks on the host itself?
>>
>
> I have a monitoring tool that accompanies my new plugin, which basically
> uses a config file that contains details about the cluster (IPs, VIPs,
> networks etc). This config file is typically created on the installer nodes
> through the deployment, Fuel Master in this case.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins] Role for Fuel Master Node

2015-11-06 Thread Evgeniy L

Great, let us know, if you have any problems.

Also for the future we have some ideas/plans to provide a way for
a plugin to retrieve any information from API.

Thanks,

On Fri, Nov 6, 2015 at 12:16 PM, Javeria Khan <javer...@plumgrid.com> wrote:

> Thank Evgeniy.  I came to the same conclusion.
>
>
> --
> Javeria
>
> On Fri, Nov 6, 2015 at 1:41 PM, Evgeniy L <e...@mirantis.com> wrote:
>
>> Javeria,
>>
>> In your case, I think it's easier to generate config on the target node,
>> using puppet for example, since the information which you may need
>> is placed in /etc/astute.yaml file. Also it may be a problem to retrieve
>> all required information about the cluster, since API is protected with
>> keystone authentication.
>>
>> Thanks,
>>
>> On Thu, Nov 5, 2015 at 5:35 PM, Javeria Khan <javer...@plumgrid.com>
>> wrote:
>>
>>> Hi Evgeniy,
>>>
>>>>
>>>> 1. what version of Fuel do you use?
>>>>
>>> Using 7.0
>>>
>>>
>>>> 2. could you please clarify what did you mean by "moving to
>>>> deployment_tasks.yaml"?
>>>>
>>> I tried changing my tasks.yaml to a deployment_tasks.yaml as the wiki
>>> suggests for 7.0. However I kept hitting issues.
>>>
>>>
>>>> 3. could you please describe your use-case a bit more? Why do you want
>>>> to run
>>>> tasks on the host itself?
>>>>
>>>
>>> I have a monitoring tool that accompanies my new plugin, which basically
>>> uses a config file that contains details about the cluster (IPs, VIPs,
>>> networks etc). This config file is typically created on the installer nodes
>>> through the deployment, Fuel Master in this case.
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Master node upgrade

2015-11-06 Thread Evgeniy L

Hi Vladimir,

Cannot say anything about 1st option, which is to use official Centos
scripts,
because I'm not familiar with the procedure, but since our installation is
not
really Centos, I have doubts that it's going to work correctly.

2nd option looks less risky. Also we should decide when to run containers
upgrade + host upgrade? Before or after new CentOS is installed? Probably
it should be done before we run backup, in order to get the latest scripts
for
backup/restore actions.

Thanks,

On Fri, Nov 6, 2015 at 1:29 PM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> Dear colleagues,
>
> At the moment I'm working on deprecating Fuel upgrade tarball. Currently,
> it includes the following:
>
> * RPM repository (upstream + mos)
> * DEB repository (mos)
> * openstack.yaml
> * version.yaml
> * upgrade script itself (+ virtualenv)
>
> Apart from upgrading docker containers this upgrade script makes copies of
> the RPM/DEB repositories and puts them on the master node naming these
> repository directories depending on what is written in openstack.yaml and
> version.yaml. My plan was something like:
>
> 1) deprecate version.yaml (move all fields from there to various places)
> 2) deliver openstack.yaml with fuel-openstack-metadata package
> 3) do not put new repos on the master node (instead we should use online
> repos or use fuel-createmirror to make local mirrors)
> 4) deliver fuel-upgrade package (throw away upgrade virtualenv)
>
> Then UX was supposed to be roughly like:
>
> 1) configure /etc/yum.repos.d/nailgun.repo (add new RPM MOS repo)
> 2) yum install fuel-upgrade
> 3) /usr/bin/fuel-upgrade (script was going to become lighter, because
> there should have not be parts coping RPM/DEB repos)
>
> However, it turned out that Fuel 8.0 is going to be run on Centos 7 and it
> is not enough to just do things which we usually did during upgrades. Now
> there are two ways to upgrade:
> 1) to use the official Centos upgrade script for upgrading from 6 to 7
> 2) to backup the master node, then reinstall it from scratch and then
> apply backup
>
> Upgrade team is trying to understand which way is more appropriate.
> Regarding to my tarball related activities, I'd say that this package based
> upgrade approach can be aligned with (1) (fuel-upgrade would use official
> Centos upgrade script as a first step for upgrade), but it definitely can
> not be aligned with (2), because it assumes reinstalling the master node
> from scratch.
>
> Right now, I'm finishing the work around deprecating version.yaml and my
> further steps would be to modify fuel-upgrade script so it does not copy
> RPM/DEB repos, but those steps make little sense taking into account Centos
> 7 feature.
>
> Colleagues, let's make a decision about how we are going to upgrade the
> master node ASAP. Probably my tarball related work should be reduced to
> just throwing tarball away.
>
>
> Vladimir Kozhukalov
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Default PostgreSQL server encoding is 'ascii'

2015-11-05 Thread Evgeniy L

Hi,

I believe we don't have any VirtualBox specific hacks, especially in terms
of
database configuration. By "development env" Vitaly meant fake UI, when
developer installs and configures the database by himself, without any iso
images, so probably his db is configured correctly with utf-8.

Also we should make sure that upgrade works correctly, after the problem is
fixed.

Thanks,

On Wed, Nov 4, 2015 at 2:30 PM, Artem Roma  wrote:

> Hi, folks!
>
> Recently I've been working on this bug [1] and have found that default
> encoding of database server used by FUEL infrastructure components
> (Nailgun, OSTF, etc) is ascii. At least this is true for environment set up
> via VirtualBox scripts. This situation may (and already does returning to
> the bug) cause obfuscating problems when dealing with non-ascii string data
> supplied by user such as names for nodes, clusters etc. Nailgun encodes
> such data in UTF-8 before sending to the database so misinterpretation by
> former while saving it is sure thing.
>
> I wonder if we have such situation on all Fuel environments or only on
> those set by VB scripts, because as for me it seems as pretty serious flaw
> in our infrastructure. It would be great to have some comments from people
> more competent in areas regarding to the matter.
>
> [1] https://bugs.launchpad.net/fuel/+bug/1472275
>
> --
> Regards!)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins] Role for Fuel Master Node

2015-11-05 Thread Evgeniy L

Hi Javeria,

As far as I know there is no way to run the task on Fuel master host itself.
Since MCollective is installed in the container and tasks get executed using
MCollective, as a workaround you may try to ssh from the container to the
host.

Also I have several additional questions:
1. what version of Fuel do you use?
2. could you please clarify what did you mean by "moving to
deployment_tasks.yaml"?
3. could you please describe your use-case a bit more? Why do you want to
run
tasks on the host itself?

Thanks,

On Wed, Nov 4, 2015 at 11:25 PM, Javeria Khan  wrote:

> Thanks Igor, Alex. Guess there isn't any support for running tasks
> directly on the Fuel Master node for now.
>
> I did try moving to deployment_tasks.yaml, however it leads to other
> issues such as "/etc/fuel/plugins// does not exist" failing on
> deployments.
>
> I'm trying to move back to using the former tasks.yaml, but the
> fuel-plugin-builder keeps looking for deployment_tasks.yaml now. There
> should be some build source list I can remove?
>
>
> --
> Javeria
>
> On Wed, Nov 4, 2015 at 12:44 PM, Aleksandr Didenko 
> wrote:
>
>> Hi,
>>
>> please note that such tasks are executed inside 'mcollective' docker
>> container, not on the Fuel master host system.
>>
>> Regards,
>> Alex
>>
>> On Tue, Nov 3, 2015 at 10:41 PM, Igor Kalnitsky 
>> wrote:
>>
>>> Hi Javeria,
>>>
>>> Try to use 'master' in 'role' field. Example:
>>>
>>> - role: 'master'
>>>   stage: pre_deployment
>>>   type: shell
>>>   parameters:
>>>   cmd: echo all > /tmp/plugin.all
>>>   timeout: 42
>>>
>>> Let me know if you need additional help.
>>>
>>> Thanks,
>>> Igor
>>>
>>> P.S: Since Fuel 7.0 it's recommended to use deployment_tasks.yaml
>>> instead of tasks.yaml. Please see Fuel Plugins wiki page for details.
>>>
>>> On Tue, Nov 3, 2015 at 10:26 PM, Javeria Khan 
>>> wrote:
>>> > Hey everyone,
>>> >
>>> > I've been working on a fuel plugin and for some reason just cant
>>> figure out
>>> > how to run a task on the fuel master node through the tasks.yaml. Is
>>> there
>>> > even a role for it?
>>> >
>>> > Something similar to what ansible does with localhost would work.
>>> >
>>> > Thanks,
>>> > Javeria
>>> >
>>> >
>>> __
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins][UX] Component registry

2015-11-02 Thread Evgeniy L

Hi,

The main reason why I think we should get all of the three states is we
don't know exactly if those plugins (which developer didn't specify) are
compatible or not, so we should not make any assumptions and prevent
the user from enabling any plugins she/he wants. The best we can do here
is to provide all of the information plugin developer knows, directly to
the user,
without us in the middle who make decisions based on incomplete data.

So lets ask plugin developer to specify a set of components which he tested
his plugin with. Plus a list of components which he tested with and he is
sure
that those are not going to working together.

On UI we can show explicitly, that this combination is tested and approved
to
be working, another combination is not working for sure (plugin developers
tested
it and explicitly specified), and there will be a lot of combination which
are going
to work together without problems, but we should say the user, that the
developer
didn't test it and he should test and use it carefully.

Thanks,

On Mon, Nov 2, 2015 at 11:22 AM, Andriy Popovych 
wrote:

> Hi fuelers,
>
> Currently we are working on feature component registry [1] which should
> help us to prevent not logical compositions of different components in
> wizard tab during cluster(environment) creation. Now we have a mechanizm
> of 'restrictions' which is not flexible for components provided by
> plugins. In our current approach we have two states for components -
> compatible/incompatible which are described in compatibility matrix
> based on OpenStack components (For example: cinder-vmware storage only
> compatible with vCetner hypervisor and should work when only KVM uses).
> In this case we allow user to choose only that components we defently
> know works well with each other. Another approach tell us to have 3
> states: compatible/incompatible/ and all other components about
> compatibility with others we know nothing. In that case we should show
> on wizard which components compatible, which not, and others which user
> can enable on his own risk. So I need your opinions: should we allow for
> user choose only that coponents which are tested and defently works or
> prevent her/him from choosing which are defently not works and means
> potentional risk of failing deployment when component about we know
> nothing isn't work together.
>
>
>
> [1] https://blueprints.launchpad.net/fuel/+spec/component-registry
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Modularization activity POC

2015-10-20 Thread Evgeniy L

Hi Mike,

1. our plan is to have working partitioning/provisioning in a couple of
weeks,
networking is more complicated and it's better to ask Vladimir/Ryan.
2, 3. here we have just some general ideas, we should have independent
releases on pypi,
each component should have responsible person (team), the same way we
do it for fuel-plugin-builder
and fuelclient. The same applies to independent docker containers.
Another thing is how
we are going to combine it in ready to use tool, there are several
ways, if we drop containers,
those will be just rpm/deb packages and dependencies between them, if
we continue using
docker containers, the simplest and the most robust way is to build a
single container which
has everything user needs.

Regarding the spec our plan is to start writing spec after we have working
POC.

Thanks,


On Tue, Oct 20, 2015 at 4:43 AM, Mike Scherbakov <mscherba...@mirantis.com>
wrote:

> This is great start, Evgeny.
> I have a few questions:
>
>1. When do you expect to have POC to show?
>2. Do you plan to put new services into separate repos?
>3. How do you plan to package those - will you create RPM package for
>each, as well as Docker container (as we have everything in containers on
>Fuel master node)
>
> These questions, of course, should be covered in spec - so may be I should
> just wait for you guys to create specs. I just think that it's very
> important to think about integration from the very beginning.
>
> Thanks,
>
> On Mon, Oct 19, 2015 at 8:54 AM Igor Kalnitsky <ikalnit...@mirantis.com>
> wrote:
>
>> Hey Evgeniy.
>>
>> This is awesome news1 I believe that microservices is way to go.
>> Despite the fact that them bring a set of classical problems (e.g.
>> duplication of domain entities) we will win more than loose. :)
>>
>> If there will be any specs or design meetings, please send me invite -
>> I'd gladly join discussions.
>>
>> Thanks,
>> Igor
>>
>>
>>
>>
>> On Wed, Oct 14, 2015 at 5:30 PM, Evgeniy L <e...@mirantis.com> wrote:
>> > Hi,
>> >
>> > We are starting Fuel modularization POC activity which is basically in
>> one
>> > sentence
>> > can be explained as "Use Fuel without Fuel", it means that we want to
>> > provide
>> > for a user a way to use some good things from Fuel, without entire
>> master
>> > node and
>> > other parts which user doesn't need.
>> >
>> > Currently we have monolithic system which includes every aspect of
>> > deployment
>> > logic, those components tightly coupled together, and there are few
>> persons
>> > who understand all the interactions between components.
>> >
>> > As a result of modularization we will get the next benefits:
>> > 1. reusability of components
>> > 1.1. it will lead to more components consumers (users)
>> > 1.2. better integration with the community (some community projects
>> might
>> >be interested in using some parts of Fuel)
>> > 2. components decoupling will lead to clear interfaces between
>> components
>> > 2.1. so it will be easier to replace some component
>> > 2.2. it will be easier to split the work between teams and it will help
>> > scale teams in
>> >a much more efficient way
>> >
>> > Here are some thing which naturally can be used separately:
>> > * network configuration (is a part of POC)
>> > * advanced partitioning/provisioning (is a part of POC)
>> > * discovery mechanism (ironic-inspector?)
>> > * power management (ironic?)
>> > * network verification
>> > * diagnostic snapshot
>> > * etc
>> >
>> > The main purpose of POC is to try to make partly working system to
>> figure
>> > out the
>> > scope of work which we will have to do upstream in order to make it
>> > production ready.
>> >
>> > Here are few basic component-specific ideas:
>> >
>> > Advanced partitioning/provisioning
>> > * split provisioning into two independent actions partitioning and
>> > provisioning
>> >   (currently we have a single call which does partitioning,
>> provisioning,
>> >post provisioning configuration)
>> > * figure out the data format (currently it's too Fuel and Cobbler
>> specific)
>> >
>> > Network configuration
>> > * CRUD api on any entity in the system (in Fuel not all of the data are
>> > exposed via api,
>> >   so user has to go and edit entries in d

Re: [openstack-dev] [Fuel][Fuel-Modularization] Proposal on Decoupling Serializers from Nailgun

2015-10-20 Thread Evgeniy L

Hi,

I have a comment regarding to when/where run translators.
I think data processing (fetching + validation + translation) should be done
as a separate stage, this way when we start deployment, we are sure
that we have everything we need to perform deployment, but I understand
that there might be exceptions and sometimes we will have to get the data
on the fly.
If we are going to put translated data into some distributed store I'm not
sure
if distributed approach for fetching and updating the data won't cause the
problems
with race conditions, in centralised approach probability to get such
problems can
be reduced.

Thanks,


On Tue, Oct 20, 2015 at 5:14 AM, Mike Scherbakov 
wrote:

> Thanks Vladimir.
> This is very important work, I'd say. I'd split it into two parts:
>
>1. Have some store where you'd dump data from serializers. Hiera
>should be able to read easily directly from this store.
>2. Refactor serializers so to get rid of single python method which
>creates data for multiple OpenStack components, and allow deployment
>engineers to easily modify code of particular piece
>
> For #1, it is important to think broadly. We want this store to be used by
> other tools which users may have (Puppet Master, Ansible, etc.) as a source
> data, and so that Fuel & other tools can coexist on the same env if really
> needed (even though I'd ideally try to avoid it).
> We need to have an abstraction layer there, so that we can have drivers
> for key-value store and for such things as Zookeeper, for instance, in the
> future. I think we need to address #1 in the first order, before going to
> #2 (if we can't do it in parallel).
>
> For #2, I think we need to consider flexibility. What if we use Ansible,
> or container for some of our services? So we need to think where we can put
> these per-component / per-task serializers, so those can be used for both
> Puppet module & something different.
>
> Also, it's interesting problem from the execution point of view. Do we run
> serialization on Fuel Master side or on slave nodes, where we install
> OpenStack? I see some issues with running it on OpenStack nodes, even
> though I like an idea of load distribution, etc. For instance, if you run
> almost all graph, and then the last task in the graph runs corresponding
> serializer - and there is a Python exception for whatever reason (user
> input leads to bug in calculation). You could get it right a way, if you
> tried to calculate it before overall deployment - but now you've been
> waiting deployment to be almost done to catch it.
>
> Thank you,
>
> On Fri, Oct 16, 2015 at 9:22 AM Vladimir Kuklin 
> wrote:
>
>> Hey, Fuelers
>>
>> TL;DR This email is about how to make
>>
>> * Intro
>> I want to bring up one of the important topics on how to make Fuel more
>> flexible. Some of you know that we have been discussing means of doing this
>> internally and now it is time to share these thoughts with all of you.
>>
>> As you could know per Evgeniy Li's message [0] we are looking forward
>> splitting Fuel (specifically it's Fuel-Web) part into set of microservices
>> each one serving their own purpose like networking configuration,
>> partitioning, etc.
>>
>>
>> And while we are working on this it seems that we need to get rid of
>> so-called Nailgun serializers that are put too close to business logic
>> engine, that have a lot of duplicating attributes; you are not able to
>> easily modify or extend them; you are not able to change their behaviour
>> even when Fuel Library is capable of doing so - everything is hardcoded in
>> Nailgun code without clear separation between business logic and actual
>> deployment workflow data generation and orchestration.
>>
>> Let me give you an example:
>>
>> * Case A. Replace Linux bridges with OVS bridges by default
>>
>> We all know that we removed OVS as much as possible from our reference
>> architecture due to its buginess. Imagine a situation when someone
>> magically fixed OVS and wants to use it as a provider for generic bonds and
>> bridge. It actually means that he needs to set default provider in
>> network_scheme for l23network puppet module to 'ovs' instead of 'lnx'.
>> Imagine, he has put this magical OVS into a package and created a plugin.
>> The problem here will be that he needs to override what network serializer
>> is sending to the nodes.
>>
>> But the problem here is that he cannot do it without editing Nailgun code
>> or override this serializer in any way.
>>
>> * Case B. Make Swift Partitions Known to Fuel Library
>>
>> Imagine, you altered the way you partition your disk in Nailgun. You
>> created a special role for swift disks which should occupy the whole disk.
>> In this case you should be able to get this info from api and feed it to
>> swift deployment task. But it is not so easy - this stuff is still
>> hardcoded in deployment serializers like {mp} field of nodes array of
>> hashes.
>>
>>

[openstack-dev] [Fuel] Modularization activity POC

2015-10-14 Thread Evgeniy L

Hi,

We are starting Fuel modularization POC activity which is basically in one
sentence
can be explained as "Use Fuel without Fuel", it means that we want to
provide
for a user a way to use some good things from Fuel, without entire master
node and
other parts which user doesn't need.

Currently we have monolithic system which includes every aspect of
deployment
logic, those components tightly coupled together, and there are few persons
who understand all the interactions between components.

As a result of modularization we will get the next benefits:
1. reusability of components
1.1. it will lead to more components consumers (users)
1.2. better integration with the community (some community projects might
   be interested in using some parts of Fuel)
2. components decoupling will lead to clear interfaces between components
2.1. so it will be easier to replace some component
2.2. it will be easier to split the work between teams and it will help
scale teams in
   a much more efficient way

Here are some thing which naturally can be used separately:
* network configuration (is a part of POC)
* advanced partitioning/provisioning (is a part of POC)
* discovery mechanism (ironic-inspector?)
* power management (ironic?)
* network verification
* diagnostic snapshot
* etc

The main purpose of POC is to try to make partly working system to figure
out the
scope of work which we will have to do upstream in order to make it
production ready.

Here are few basic component-specific ideas:

Advanced partitioning/provisioning
* split provisioning into two independent actions partitioning and
provisioning
  (currently we have a single call which does partitioning, provisioning,
   post provisioning configuration)
* figure out the data format (currently it's too Fuel and Cobbler specific)

Network configuration
* CRUD api on any entity in the system (in Fuel not all of the data are
exposed via api,
  so user has to go and edit entries in db manually)
* no constraints regarding to network topology (in Fuel there are a lot of
hardcoded
  assumptions)

Node hardware discovery
* should be able to support different source drivers at the same time
   (csv file, fuel-nailgun-agent, CMDB, dhcp, cobbler etc)
* versioning of HW information (required for life cycle management)
* notification about changes in hardware or about node statuses
  (required for life cycle management)

Common requirements for components:
* every component must follow OpenStack coding standards when
  we start working upstream after POC (so there shouldn't be a question
  what to use pecan of flask)
* Fuel specific interfaces should be implemented as drivers
* this one is obvious, but currently one of the most problematic parts in
Fuel,
  HW gets changed, interface can be replaced, disk can be removed,
  component should have a way to handle it

Technically speaking, we want to have separate services for every component,
it is one of the technical ways to force components to have clear
interfaces.

Other architecture decision which we want to try to solve is extendability,
currently we have a problem that all of the logic which is related to
deployment
data is hardcoded in Nailgun. Plugins should be able to retrieve any data it
needs from the system, in order to perform plugin deployment, these data
should
be retrieved using some interface, and we already have interface where we
can
provide stability and versioning, it's REST API. So basically deployment
should
consist of two steps first is to retrieve all required data from any source
it needs,
and after that send data to the nodes and start deployment.

If you have any questions/suggestion don't hesitate to ask.

Thanks,
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Plugins related functionality in Fuel Client

2015-10-09 Thread Evgeniy L

>> I’d say even if it will be a separate service it’s better to proxy
requests through Nailgun’s API to have a single entry point.

I don't think that application such as Nailgun should be responsible
for proxying
requests, we solved similar problem for OSTF with adding proxy rule in
Nginx.

Thanks,

On Fri, Oct 9, 2015 at 11:45 AM, Roman Prykhodchenko <m...@romcheg.me> wrote:

> I’d say even if it will be a separate service it’s better to proxy
> requests through Nailgun’s API to have a single entry point.
>
> 9 жовт. 2015 р. о 10:23 Evgeniy L <e...@mirantis.com> написав(ла):
>
> Hi,
>
> +1, but I think it's better to spawn separate service, instead of adding
> it to Nailgun.
>
> Thanks,
>
> On Fri, Oct 9, 2015 at 1:40 AM, Roman Prykhodchenko <m...@romcheg.me> wrote:
>
>> Folks,
>>
>> it’s time to speak about Fuel Plugins and the way they are managed.
>>
>> Currently we have some methods in Fuel Client that allow to install,
>> remove and do some other things to plugins. Everything looks great except
>> that functionality requires Fuel Client to be installed on a master node
>> and be running under a root user. It’s time for us to grow up and realize
>> that nothing can require Fuel Client to be installed on a specific computer
>> and of course we cannot require root permissions for any actions.
>>
>> I’d like to move all that code to Nailgun, utilizing mules and hide it
>> behind Nailgun’s API as soon as possible. For that I filed a bug [1] and
>> I’d like to ask Fuel Enhancements subgroup of developers to take a close
>> look at it.
>>
>>
>> 1. https://bugs.launchpad.net/fuel/+bug/1504338
>>
>>
>> - romcheg
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> <http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Plugins related functionality in Fuel Client

2015-10-09 Thread Evgeniy L

Hi,

+1, but I think it's better to spawn separate service, instead of adding it
to Nailgun.

Thanks,

On Fri, Oct 9, 2015 at 1:40 AM, Roman Prykhodchenko  wrote:

> Folks,
>
> it’s time to speak about Fuel Plugins and the way they are managed.
>
> Currently we have some methods in Fuel Client that allow to install,
> remove and do some other things to plugins. Everything looks great except
> that functionality requires Fuel Client to be installed on a master node
> and be running under a root user. It’s time for us to grow up and realize
> that nothing can require Fuel Client to be installed on a specific computer
> and of course we cannot require root permissions for any actions.
>
> I’d like to move all that code to Nailgun, utilizing mules and hide it
> behind Nailgun’s API as soon as possible. For that I filed a bug [1] and
> I’d like to ask Fuel Enhancements subgroup of developers to take a close
> look at it.
>
>
> 1. https://bugs.launchpad.net/fuel/+bug/1504338
>
>
> - romcheg
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][PTL] PTL Candidates Q Session

2015-10-02 Thread Evgeniy L

Hi Mike,

According to the description of the role, I wouldn't say that the role is
less architectural than
political, since PTL should review designs and resolve conflicts between
cores (which are
usually technical), PTL should also have strong skills in software
architecture, and understanding
of what Fuel should look like.

Thanks,

On Thu, Oct 1, 2015 at 11:32 PM, Mike Scherbakov 
wrote:

> > we may mix technical direction / tech debt roadmap and process,
> political, and people management work of PTL.
> sorry, of course I meant that we rather should NOT mix these things.
>
> To make my email very short, I'd say PTL role is more political and
> process-wise rather than architectural.
>
> On Wed, Sep 30, 2015 at 5:48 PM Mike Scherbakov 
> wrote:
>
>> Vladimir,
>> we may mix technical direction / tech debt roadmap and process,
>> political, and people management work of PTL.
>>
>> PTL definition in OpenStack [1] reflects many things which PTL becomes
>> responsible for. This applies to Fuel as well.
>>
>> I'd like to reflect some things here which I'd expect PTL doing, most of
>> which will intersect with [1]:
>> - Participate in cross-project initiatives & resolution of issues around
>> it. Great example is puppet-openstack vs Fuel [2]
>> - Organize required processes around launchpad bugs & blueprints
>> - Personal personal feedback to Fuel contributors & public suggestions
>> when needed
>> - Define architecture direction & review majority of design specs. Rely
>> on Component Leads and Core Reviewers
>> - Ensure that roadmap & use cases are aligned with architecture work
>> - Resolve conflicts between core reviewers, component leads. Get people
>> to the same page
>> - Watch for code review queues and quality of reviews. Ensure discipline
>> of code review.
>> - Testing / coverage have to be at the high level
>>
>> Considering all above, contributors actually have been working with all
>> of us and know who could be better handling such a hard work. I don't think
>> special Q is needed. If there are concerns / particular process/tech
>> questions we'd like to discuss - those should be just open as email threads.
>>
>> [1] https://wiki.openstack.org/wiki/PTL_Guide
>> [2]
>> http://lists.openstack.org/pipermail/openstack-dev/2015-June/066685.html
>>
>> Thank you,
>>
>> On Tue, Sep 29, 2015 at 3:47 AM Vladimir Kuklin 
>> wrote:
>>
>>> Folks
>>>
>>> I think it is awesome we have three candidates for PTL position in Fuel.
>>> I read all candidates' emails (including mine own several times :-) ) and I
>>> got a slight thought of not being able to really differentiate the
>>> candidates platforms as they are almost identical from the high-level point
>>> of view. But we all know that the devil is in details. And this details
>>> will actually affect project future.
>>>
>>> Thus I thought about Q session at #fuel-dev channel in IRC. I think
>>> that this will be mutually benefitial for everyone to get our platforms a
>>> little bit more clear.
>>>
>>> Let's do it before or right at the start of actual voting so that our
>>> contributors can make better decisions based on this session.
>>>
>>> I suggest the following format:
>>>
>>> 1) 3 questions from electorate members - let's put them onto an etherpad
>>> 2) 2 questions from a candidate to his opponents (1 question per
>>> opponent)
>>> 3) external moderator - I suppose, @xarses as our weekly meeting
>>> moderator could help us
>>> 4) time and date - Wednesday or Thursday comfortable for both timezones,
>>> e.g. after 4PM UTC or right after fuel weekly meeting.
>>>
>>> What do you think, folks?
>>>
>>> --
>>> Yours Faithfully,
>>> Vladimir Kuklin,
>>> Fuel Library Tech Lead,
>>> Mirantis, Inc.
>>> +7 (495) 640-49-04
>>> +7 (926) 702-39-68
>>> Skype kuklinvv
>>> 35bk3, Vorontsovskaya Str.
>>> Moscow, Russia,
>>> www.mirantis.com 
>>> www.mirantis.ru
>>> vkuk...@mirantis.com
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>> --
>> Mike Scherbakov
>> #mihgen
>>
> --
> Mike Scherbakov
> #mihgen
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Bugfixing status. 12 weeks before SCF.

2015-10-02 Thread Evgeniy L

Hi Dmitry,

Thanks for the information.

>> My personal opinion is that we can ignore our medium-priority technical
debt bugs for now. We should fix them but there is nothing that cannot be
postponed here. We will continue fixing them in background. The only
exception here should be bugs related to alignment with global
requirements, tox and oslo-related changes. We definitely should fix this
stuff.

I think we shouldn't ignore medium priority tech debt bugs now, the
beginning of the release
cycle is the best time to fix them.

Thanks,

On Thu, Oct 1, 2015 at 9:17 PM, Dmitry Pyzhov  wrote:

> Guys,
>
> I was not able to participate in our weekly IRC meeting. So I'd like to
> share our bug status for 8.0 release with offline e-mail.
>
> We have 494 Fuel bugs on Launchpad. This number can be splitted into
> several piles.
>
> 1) Critical and High priority bugs. We have 48 of them now. 2 in UI, 31 in
> python, 15 in library. Here is our focus and we are working on reducing the
> numbers.
> 2) Medium/Low/Wishlist priority bugs. We have 241 bug. 72 in UI, 119 in
> python, 50 in library.
> 3) Features reported as bugs and bugs that can be fixed only by
> implementing new blueprints. We have 133 of them. 3 in UI, 106 in python
> and 24 in library. These bugs are marked with 'feature', 'covered-by-bp'
> and 'need-bp' tags. Numbers look scary but only 40 of them have high and
> critical priority.
> 4) Technical debt. Things that we should do better from developer's point
> of view. 72 bugs in total. 60 in python, 12 in library. They are marked
> with 'tech-debt' tag.
>
> My personal opinion is that we can ignore our medium-priority technical
> debt bugs for now. We should fix them but there is nothing that cannot be
> postponed here. We will continue fixing them in background. The only
> exception here should be bugs related to alignment with global
> requirements, tox and oslo-related changes. We definitely should fix this
> stuff.
>
> You can see that we have big demand for python developers. Here is my
> early estimation. With current pace we can fix all existing library bugs in
> 8.0. Also we can fix all existing high priority bugs in python. It includes
> technical debt and maybe feature-bugs. It looks like we are able to fix
> about half of medium priority python bugs. I don't have any estimations for
> medium priority feature-bugs in python. And I'd prefer to be pessimistic
> here. Also we will fix very small number of medium priority technical debt
> bugs.
>
> There is a good chance that number of incoming bugs will became smaller
> over time and we will fix most of existing medium priority python bugs.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Remove nova-network as a deployment option in Fuel?

2015-10-01 Thread Evgeniy L

Hi,

Just a small note, we shouldn't remove it completely from Nailgun codebase,
because we still have old environments to support, we should remove it from
newest releases only.

Thanks,

On Thu, Oct 1, 2015 at 1:10 AM, Mike Scherbakov 
wrote:

> Hi team,
> where do we stand with it now? I remember there was a plan to remove
> nova-network support in 7.0, but we've delayed it due to vcenter/dvr or
> something which was not ready for it.
>
> Can we delete it now? The early in the cycle we do it, the easier it will
> be.
>
> Thanks!
> --
> Mike Scherbakov
> #mihgen
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Code review process in Fuel and related issues

2015-09-28 Thread Evgeniy L

Hi Mike, thanks, now it's clear.

On Thu, Sep 3, 2015 at 9:19 AM, Mike Scherbakov 
wrote:

> Thank you all for the feedback.
>
>
> Dims -
>
> > 1) I'd advise to codify a proposal in fuel-specs under a 'policy'
> directory
>
> I think it's great idea and I'll do it.
>
>
> > 2) We don't have SME terminology, but we do have Maintainers both in
> oslo-incubator
>
> I like "Maintainers" more than SMEs, thank you for suggestion. I'd switch
> SME -> Maintainer everywhere.
>
>
> > 3) Is there a plan to split existing repos to more repos? Then each repo
> can have a core team (one core team for one repo), PTL takes care of all
> repos and MAINTAINERS take care of directories within a repo. That will
> line up well with what we are doing elsewhere in the community (essentially
> "Component Lead" is a core team which may not be a single person).
>
> That's the plan, with one difference though. According to you, there is a
> core team per repo without a lead identified. In my proposal, I'd like to
> ensure that we always choose a lead by the process of voting. I'd like to
> have voting process of identifying a component lead. It has to be fair
> process.
>
>
> > We do not have a concept of SLA anywhere that i know of, so it will have
> to be some kind of social consensus and not a real carrot/stick.
>
> > As for me the idea of SLA contradicts to qualitative reviews.
>
> I'm not thinking about carrot or stick here. I'd like to ensure that core
> reviewers and component leads are targeted to complete reviews within a
> certain time. If it doesn't happen, then patchset needs to be discussed
> during IRC meeting if we can delegate some testing, etc. to someone. If
> there are many patchsets out of SLA, then we'd consider other changes
> (decide to drop something from the release so to free up resources or
> something else).
>
> We had a problem in the past, when we would not pay attention to a patch
> proposed by someone before it's being escalated. I'm suggesting a solution
> for that problem. SLA is the contract between contributor and reviewer, and
> both would have same expectations on how long would it take to review the
> patch. Without expectations aligned, contributors can get upset easily.
> They may expect that their code will be reviewed and merged within hours,
> while it fact it's days. I'm not even talking about patches which can be
> forgotten and hang in the queue for months...
>
>
> > If we succeed in reducing the load on core reviewers, it will mean that
> core reviewers will do less code reviews. This could lead to core reviewer
> demotion.
>
> I expect that there will be a drop in code reviews being done by core
> reviewers team. This is the point actually - do less reviews, but do it
> more thoroughly. Don't work on patches which have easy mistakes, as those
> should be cought by maintainers, before these patches come to the core
> reviewer's plate. I don't think though that it will lead to core reviewer
> "demotion". Still, you will be doing many reviews - just less than before,
> and other who did a little - will do more reviews, potentially becoming
> joining a core team later.
>
>
> > It would be nice if Jenkins could add reviewers after CI +1, or we can
> use gerrit dashboard for SMEs to not waste their time on review that has
> not yet passed CI and does not have +1 from other reviewers.
>
> This is good suggestion. I agree.
>
>
> > AFAIK Boris Pavlovic introduced some scripts
>
> > in Rally which do basic preliminary check of review message, checking
>
> > that it's formally correct.
>
> Thanks Igor, I believe this can be applied as well.
>
>
> > Another thing is I got a bit confused by the difference between Core
> Reviewer and Component Lead,
>
> > aren't those the same persons? Shouldn't every Core Reviewer know the
> architecture, best practises
>
> > and participate in design architecture sessions?
>
> Component Lead is being elected by core reviewers team as the lead. So
> it's just another core reviewer / architect, but with the right to have a
> final word. Also, for large parts like fuel-library / nailgun, I'd expect
> this person to be free from any feature-work. For smaller things, like
> network verifier, I don't think that we'd need to have dedicated component
> owner who will be free from any feature work.
>
>
> Igor K., Evgeny L, did I address your questions regarding SLA and
> component lead vs core reviewer?
>
> Thank you,
>
> On Wed, Sep 2, 2015 at 9:28 AM Jay Pipes  wrote:
>
>> On 09/02/2015 08:45 AM, Igor Kalnitsky wrote:
>> >> I think there's plenty of examples of people in OpenStack projects
>> >> that both submit code (and lead features) that also do code review
>> >> on a daily basis.
>> >
>> > * Do these features huge?
>>
>> Yes.
>>
>> > * Is their code contribution huge or just small patches?
>>
>> Both.
>>
>> > * Did they get to master before FF?
>>
>> Yes.
>>
>> > * How many intersecting features OpenStack projects have

Re: [openstack-dev] [Fuel] Code review process in Fuel and related issues

2015-08-27 Thread Evgeniy L

 - SME reviews the code within SLA, which should be defined per component

Also I would like to add, that I'm not against of metrics, we can collect
metrics, in order to figure out if some improvement in the process helped
to speed up reviews, but asking Cores/SMEs to do the job faster will
definitely
affect quality.

So the suggestion is to collect metrics *only* to prove that some
improvement
in the process helped or didn't help.

On Thu, Aug 27, 2015 at 5:58 PM, Evgeniy L e...@mirantis.com wrote:

 Hi Mike,

 I have several comments.

  SLA should be the driver of doing timely reviews, however we can’t
 allow to fast-track code into master suffering quality of review ...

 As for me the idea of SLA contradicts to qualitative reviews.

 Another thing is I got a bit confused by the difference between Core
 Reviewer and Component Lead,
 aren't those the same persons? Shouldn't every Core Reviewer know the
 architecture, best practises
 and participate in design architecture sessions?

  - If core reviewer has not landed the code yet, Component Lead merges
 patch within SLA defined (or declines to merge and provides explanation as
 part of review).

 For example here as far as I'm concerned Component Lead is Core reviewer,
 since
 he has permissions to merge.

 Thanks,


 On Wed, Aug 19, 2015 at 11:31 AM, Mike Scherbakov 
 mscherba...@mirantis.com wrote:

 Hi all,
 let's discuss code review process in Fuel and what we can improve. For
 those who want to just have a quick context of this email, please check out
 presentation slides [5].

 ** Issues **
 Depending on a Fuel subproject, I'm aware of two buckets of issues with
 code review in Fuel:
 a) It is hard to get code reviewed and merged
 b) Quality of code review itself could be better

 First bucket:
 1) It is hard to find subject matter experts who can help and core
 reviewers for the area of code, especially if you are new to the project
 2) Contributor sometimes receives contradicting opinions from other
 reviewers, including cores
 3) Assigned / responsible core reviewer is needed for a feature in order
 to help in architectural negotiations, guiding through, landing the code
 into master
 4) Long wait time for getting code reviewed

 Quality-related items:
 5) Not thorough enough, full review in one shot. For example, reviewer
 can put -1 due to missed comma, but do not notice major gap in the code.
 It leads to many patch sets, and demotivation of contributors
 6) Some of the core reviewers decreased their involvement, and so number
 of reviews has dropped dramatically. However, they still occasionally merge
 code. I propose to remove these cores, and get them back if their
 involvement is increased back again (I very rarely merge code, but I'm one
 of those to be removed from cores). This is standard practice in OpenStack
 community as well, see Neutron as example [4, line 270].
 7) As a legacy of the past, we still have old core reviewers being able
 to merge code in all Fuel repos. All new cores have core rights only for
 single repo, which is their primary area of expertise. For example, core
 team size for fuel-library is adidenko + whole fuel-core group [7]. In
 fact, there are just 4 trusted or real core reviewers in fuel-library,
 not the whole fuel-core group.

 These problems are not new to OpenStack and open source in general. You
 can find discussions about same and similar issues in [1], [2], [3].


 ** Analysis of data **
 In order to understand what can be improved, I mined the data at first.
 Main source of information was stackalytics.com. Please take a look at
 few graphs on slides 4-7 [5], built based on data from stackalytics. Major
 conclusions from these graphs:
 1) Rather small number of core reviewers (in comparison with overall
 number of contributors) reviewing 40-60% of patch sets, depending on repo
 (40% fuel-library, 60% fuel-web). See slide #4.
 2) Load on core reviewers in Fuel team is higher in average, if you
 compare it with some other OpenStack projects. Average load on core
 reviewer across Nova, Keystone, Neutron and Cinder is 2.5 reviews a day. In
 Fuel though it is 3.6 for fuel-web and 4.6 for fuel-library. See slide #6.
 3) Statistics on how fast feedback on code proposed is provided:
 - fuel-library: 2095 total reviews in 30 days [13], 80 open reviews,
 average wait time for reviewer - 1d 1h [12]
 - fuel-web: 1789 total reviews in 30 days [14], 52 open reviews, average
 wait time for reviewer - 1d 17h [15]

 There is no need to have deep analysis on whether we have well defined
 areas of ownership in Fuel components or not: we don’t have it formally
 defined, and it’s not documented anywhere. So, finding a right core
 reviewer can be challenging task for a new contributor to Fuel, and this
 issue has to be addressed.


 ** Proposed solution **
 According to stackalytics, for the whole fuel-group we had 262 reviewers
 with 24 core reviewers for the past 180 days [19]. I think that these
 numbers can

Re: [openstack-dev] [Fuel] Code review process in Fuel and related issues

2015-08-27 Thread Evgeniy L

Hi Mike,

I have several comments.

 SLA should be the driver of doing timely reviews, however we can’t allow
to fast-track code into master suffering quality of review ...

As for me the idea of SLA contradicts to qualitative reviews.

Another thing is I got a bit confused by the difference between Core
Reviewer and Component Lead,
aren't those the same persons? Shouldn't every Core Reviewer know the
architecture, best practises
and participate in design architecture sessions?

 - If core reviewer has not landed the code yet, Component Lead merges
patch within SLA defined (or declines to merge and provides explanation as
part of review).

For example here as far as I'm concerned Component Lead is Core reviewer,
since
he has permissions to merge.

Thanks,


On Wed, Aug 19, 2015 at 11:31 AM, Mike Scherbakov mscherba...@mirantis.com
wrote:

 Hi all,
 let's discuss code review process in Fuel and what we can improve. For
 those who want to just have a quick context of this email, please check out
 presentation slides [5].

 ** Issues **
 Depending on a Fuel subproject, I'm aware of two buckets of issues with
 code review in Fuel:
 a) It is hard to get code reviewed and merged
 b) Quality of code review itself could be better

 First bucket:
 1) It is hard to find subject matter experts who can help and core
 reviewers for the area of code, especially if you are new to the project
 2) Contributor sometimes receives contradicting opinions from other
 reviewers, including cores
 3) Assigned / responsible core reviewer is needed for a feature in order
 to help in architectural negotiations, guiding through, landing the code
 into master
 4) Long wait time for getting code reviewed

 Quality-related items:
 5) Not thorough enough, full review in one shot. For example, reviewer can
 put -1 due to missed comma, but do not notice major gap in the code. It
 leads to many patch sets, and demotivation of contributors
 6) Some of the core reviewers decreased their involvement, and so number
 of reviews has dropped dramatically. However, they still occasionally merge
 code. I propose to remove these cores, and get them back if their
 involvement is increased back again (I very rarely merge code, but I'm one
 of those to be removed from cores). This is standard practice in OpenStack
 community as well, see Neutron as example [4, line 270].
 7) As a legacy of the past, we still have old core reviewers being able to
 merge code in all Fuel repos. All new cores have core rights only for
 single repo, which is their primary area of expertise. For example, core
 team size for fuel-library is adidenko + whole fuel-core group [7]. In
 fact, there are just 4 trusted or real core reviewers in fuel-library,
 not the whole fuel-core group.

 These problems are not new to OpenStack and open source in general. You
 can find discussions about same and similar issues in [1], [2], [3].


 ** Analysis of data **
 In order to understand what can be improved, I mined the data at first.
 Main source of information was stackalytics.com. Please take a look at
 few graphs on slides 4-7 [5], built based on data from stackalytics. Major
 conclusions from these graphs:
 1) Rather small number of core reviewers (in comparison with overall
 number of contributors) reviewing 40-60% of patch sets, depending on repo
 (40% fuel-library, 60% fuel-web). See slide #4.
 2) Load on core reviewers in Fuel team is higher in average, if you
 compare it with some other OpenStack projects. Average load on core
 reviewer across Nova, Keystone, Neutron and Cinder is 2.5 reviews a day. In
 Fuel though it is 3.6 for fuel-web and 4.6 for fuel-library. See slide #6.
 3) Statistics on how fast feedback on code proposed is provided:
 - fuel-library: 2095 total reviews in 30 days [13], 80 open reviews,
 average wait time for reviewer - 1d 1h [12]
 - fuel-web: 1789 total reviews in 30 days [14], 52 open reviews, average
 wait time for reviewer - 1d 17h [15]

 There is no need to have deep analysis on whether we have well defined
 areas of ownership in Fuel components or not: we don’t have it formally
 defined, and it’s not documented anywhere. So, finding a right core
 reviewer can be challenging task for a new contributor to Fuel, and this
 issue has to be addressed.


 ** Proposed solution **
 According to stackalytics, for the whole fuel-group we had 262 reviewers
 with 24 core reviewers for the past 180 days [19]. I think that these
 numbers can be considered as high enough in order to think about structure
 in which code review process would be transparent, understandable and
 scalable.

 Let’s first agree on the terminology which I’d like to use. It can take
 pages of precise definitions, however in this email thread I’d like to
 focus on code review process more, and hopefully high level description of
 roles would be enough for now.
 - Contributor: new contributor, who doesn’t work on Fuel regularly and
 doesn’t know team structure (or full time Fuel developer, who

Re: [openstack-dev] [Fuel] SSL keys saving

2015-08-24 Thread Evgeniy L

Hi,

Changes in JS will be required anyway. Because currently it sends the data
in cluster attributes dictionary. So it should send the data to some
specific url.

Regarding to authentication we had similar problem with diagnostic snapshot
[1],
you still can perform authentication in Nailgun and then Nginx will
continue to
perform the action.

You can get cluster's ID from nginx, it will be something like:

location ~ ^/api/clusters/(?cluster_id[0-9]+)/upload_key {
upload_store /var/lib/fuel/ssl_keys/$cluster_id

}

Thanks,

[1]
https://github.com/stackforge/fuel-specs/blob/5409aaca363dccf8e7598fc7995ce8ade1e68ebc/specs/7.0/snapshot-download-with-auth.rst#proposed-change

On Mon, Aug 24, 2015 at 3:31 PM, Stanislaw Bogatkin sbogat...@mirantis.com
wrote:

I preparing patches for library, but there was raised another question:
how should we saving keypairs from UI? I see there two options:

1. It will be done via UI (JS?) that will post file to special URL and
then it will be saved via nginx HttpUploadModule. Caveats here:
- I don't know how we will deal with authentication. How nginx should
understand that this file got from trusted source?
- As we should place keys in different places for different clusters, some
handler should so it, we cannot save files with pure nginx. Who will write
this handler and maintain it?

2. It will be done via Nailgun that will store files directly in FS.
Caveats here:
- How about speed/locks? Now we store just small files with certificates
(4 kb in most of cases), but in future we, maybe, will need to save big
files. Will a solution with Nailgun work okay?
- Logs. As I understand, if file will be saved by nailgun API, it will be
saved in logs and we need some tool to cut it out from there.

So, which way seems better for you, folks?

On Fri, Aug 21, 2015 at 1:59 PM, Adam Heczko ahec...@mirantis.com wrote:

Sorry in understood incorrectly - using HTTP/Web IMO usually makes kind
of overhead if designed from the beginning.
If there are some HTTP authentication/CSR request/key management
mechanisms already in place, of course there is no overhead.

Regards,

On Fri, Aug 21, 2015 at 12:43 PM, Evgeniy L e...@mirantis.com wrote:

Hi Adam,

I'm not sure if understand you correctly, what do you mean by overhead
for
certificate provisioning? We already have all the mechanisms in place
in order
to provision certificates, the point is currently with user's
certificates we work in
absolutely different way and store them in absolutely different place.
And this
way leads to huge problems.

Thanks,

On Fri, Aug 21, 2015 at 1:33 PM, Adam Heczko ahec...@mirantis.com
wrote:

Hi Evgeniy,
what you've proposed is all right, although it adds some overhead for
certificate provisioning.
In fact, to do it right we should probably define REST API for
provisioning certificates.
I'm rather for simplified approach, consisting of Shell / Puppet
scripts for certificate upload/management.

Regards,

On Fri, Aug 21, 2015 at 12:20 PM, Evgeniy L e...@mirantis.com wrote:

Hi Stanislaw,

I agree that user's certificates mustn't be saved in Nailgun database,
in cluster attributes,
in this case it can be seen in all the logs, which is terrible
security problem,
and we already have a place where we keep auto-generated certificates
and
ssh-keys, and those are copied to specific nodes by Astute.

So UI should send the file to specific URL, Nginx should be configured
to send auth
request to backend, after request is authorised, Nginx should save the
file into predefined
directory, the same which we use for autogenerated certificates, in
this case such
tool as OSTF can take certificates from a single place.

Thanks,

On Fri, Aug 21, 2015 at 12:10 PM, Stanislaw Bogatkin
sbogat...@mirantis.com wrote:

Hi folks.

Today I want to discuss the way we save SSL keys for Fuel
environments. As you maybe know we have 2 ways to get a key:
a. Generate it by Fuel (self-signed certificate will be created in
this case). In this case we will generate private key, csr and crt in a
pre-deployment hook on master node and then copy keypair to the nodes
which
needed it.

b. Get a pre-generated keypair from user. In this case user should
create keypair by himself and then upload it through Fuel UI settings
tab.
In this case keypair will be saved in nailgun database and then will
serialized into astute.yaml on cluster nodes, pulled from it by puppet
and
saved into a file.

Re: [openstack-dev] [Fuel] SSL keys saving

2015-08-24 Thread Evgeniy L

Hi John,

I agree, that we didn't have these problems if we used Barbican and looks
like it's a good tool for our needs. But since we are in Hard Code Freeze
we should fix it somehow without introducing big changes in our current
architecture.

But it would be a nice to see it as an improvement in 8.0 release.

Thanks,

On Mon, Aug 24, 2015 at 3:57 PM, John Dennis jden...@redhat.com wrote:

On 08/21/2015 05:10 AM, Stanislaw Bogatkin wrote:

Hi folks.

Today I want to discuss the way we save SSL keys for Fuel environments.
As you maybe know we have 2 ways to get a key:
a. Generate it by Fuel (self-signed certificate will be created in this
case). In this case we will generate private key, csr and crt in a
pre-deployment hook on master node and then copy keypair to the nodes
which needed it.

b. Get a pre-generated keypair from user. In this case user should
create keypair by himself and then upload it through Fuel UI settings
tab. In this case keypair will be saved in nailgun database and then
will serialized into astute.yaml on cluster nodes, pulled from it by
puppet and saved into a file.

Second way has some flaws:
1. We already have some keys for nodes and we store them on master node.
Store keys in different places is bad, cause:
1.1. User experience - user should remember that in some cases keys will
be store in FS and in some other cases - in DB.
1.2. It brings problems for implementation in other different places -
for example, we need to get certificate for properly run OSTF tests and
now we should implement two different ways to deliver that certificate
to OSTF container. The same for fuel-cli - we should somehow get
certificate from DB and place it in FS to use it.
2. astute.yaml is similar for all nodes. Not all of nodes needs to have
private key, but now we cannot control this.
3. If keypair data serializes into astute.yaml it means than that data
automatically will be fetched when diagnostic snapshot will created. So
in some cases in can lead to security vulnerability, or we will must to
write another crutch to cut it out of diagnostic snapshot.

So I propose to get rid of saving keypair in nailgun database and
implement a way to always saving it to local FS on master node. We need
to implement next items:

- Change UI logic that saving keypair into DB to logic that will save it
to local FS
- Implement according fixes in fuel-library

I have not been following this thread nor I do I know or understand all
your requirements but I wanted to bring to your attention the fact
OpenStack has a project called Barbican whose primary purpose is to safely
store keys, plus it has many other features for handling keys. Key handling
is tricky so rather than try to design something yourself perhaps it might
make sense to leverage the Barbican work.

https://wiki.openstack.org/wiki/Barbican/Incubation

http://www.rackspace.com/blog/keeping-openstack-secrets-safe-with-barbican/

http://www.slideshare.net/jarito030506/barbican-10-open-source-key-management-for-openstack

--
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] SSL keys saving

2015-08-21 Thread Evgeniy L

Hi Adam,

I'm not sure if understand you correctly, what do you mean by overhead for
certificate provisioning? We already have all the mechanisms in place in
order
to provision certificates, the point is currently with user's certificates
we work in
absolutely different way and store them in absolutely different place. And
this
way leads to huge problems.

Thanks,

On Fri, Aug 21, 2015 at 1:33 PM, Adam Heczko ahec...@mirantis.com wrote:

 Hi Evgeniy,
 what you've proposed is all right, although it adds some overhead for
 certificate provisioning.
 In fact, to do it right we should probably define REST API for
 provisioning certificates.
 I'm rather for simplified approach, consisting of Shell / Puppet scripts
 for certificate upload/management.

 Regards,

 A.


 On Fri, Aug 21, 2015 at 12:20 PM, Evgeniy L e...@mirantis.com wrote:

 Hi Stanislaw,

 I agree that user's certificates mustn't be saved in Nailgun database, in
 cluster attributes,
 in this case it can be seen in all the logs, which is terrible security
 problem,
 and we already have a place where we keep auto-generated certificates and
 ssh-keys, and those are copied to specific nodes by Astute.

 So UI should send the file to specific URL, Nginx should be configured to
 send auth
 request to backend, after request is authorised, Nginx should save the
 file into predefined
 directory, the same which we use for autogenerated certificates, in this
 case such
 tool as OSTF can take certificates from a single place.

 Thanks,

 On Fri, Aug 21, 2015 at 12:10 PM, Stanislaw Bogatkin 
 sbogat...@mirantis.com wrote:

 Hi folks.

 Today I want to discuss the way we save SSL keys for Fuel environments.
 As you maybe know we have 2 ways to get a key:
 a. Generate it by Fuel (self-signed certificate will be created in this
 case). In this case we will generate private key, csr and crt in a
 pre-deployment hook on master node and then copy keypair to the nodes which
 needed it.

 b. Get a pre-generated keypair from user. In this case user should
 create keypair by himself and then upload it through Fuel UI settings tab.
 In this case keypair will be saved in nailgun database and then will
 serialized into astute.yaml on cluster nodes, pulled from it by puppet and
 saved into a file.

 Second way has some flaws:
 1. We already have some keys for nodes and we store them on master node.
 Store keys in different places is bad, cause:
 1.1. User experience - user should remember that in some cases keys will
 be store in FS and in some other cases - in DB.
 1.2. It brings problems for implementation in other different places -
 for example, we need to get certificate for properly run OSTF tests and now
 we should implement two different ways to deliver that certificate to OSTF
 container. The same for fuel-cli - we should somehow get certificate from
 DB and place it in FS to use it.
 2. astute.yaml is similar for all nodes. Not all of nodes needs to have
 private key, but now we cannot control this.
 3. If keypair data serializes into astute.yaml it means than that data
 automatically will be fetched when diagnostic snapshot will created. So in
 some cases in can lead to security vulnerability, or we will must to write
 another crutch to cut it out of diagnostic snapshot.


 So I propose to get rid of saving keypair in nailgun database and
 implement a way to always saving it to local FS on master node. We need to
 implement next items:

 - Change UI logic that saving keypair into DB to logic that will save it
 to local FS
 - Implement according fixes in fuel-library


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Adam Heczko
 Security Engineer @ Mirantis Inc.

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] SSL keys saving

2015-08-21 Thread Evgeniy L

Hi Stanislaw,

I agree that user's certificates mustn't be saved in Nailgun database, in
cluster attributes,
in this case it can be seen in all the logs, which is terrible security
problem,
and we already have a place where we keep auto-generated certificates and
ssh-keys, and those are copied to specific nodes by Astute.

So UI should send the file to specific URL, Nginx should be configured to
send auth
request to backend, after request is authorised, Nginx should save the file
into predefined
directory, the same which we use for autogenerated certificates, in this
case such
tool as OSTF can take certificates from a single place.

Thanks,

On Fri, Aug 21, 2015 at 12:10 PM, Stanislaw Bogatkin sbogat...@mirantis.com
 wrote:

 Hi folks.

 Today I want to discuss the way we save SSL keys for Fuel environments. As
 you maybe know we have 2 ways to get a key:
 a. Generate it by Fuel (self-signed certificate will be created in this
 case). In this case we will generate private key, csr and crt in a
 pre-deployment hook on master node and then copy keypair to the nodes which
 needed it.

 b. Get a pre-generated keypair from user. In this case user should create
 keypair by himself and then upload it through Fuel UI settings tab. In this
 case keypair will be saved in nailgun database and then will serialized
 into astute.yaml on cluster nodes, pulled from it by puppet and saved into
 a file.

 Second way has some flaws:
 1. We already have some keys for nodes and we store them on master node.
 Store keys in different places is bad, cause:
 1.1. User experience - user should remember that in some cases keys will
 be store in FS and in some other cases - in DB.
 1.2. It brings problems for implementation in other different places - for
 example, we need to get certificate for properly run OSTF tests and now we
 should implement two different ways to deliver that certificate to OSTF
 container. The same for fuel-cli - we should somehow get certificate from
 DB and place it in FS to use it.
 2. astute.yaml is similar for all nodes. Not all of nodes needs to have
 private key, but now we cannot control this.
 3. If keypair data serializes into astute.yaml it means than that data
 automatically will be fetched when diagnostic snapshot will created. So in
 some cases in can lead to security vulnerability, or we will must to write
 another crutch to cut it out of diagnostic snapshot.


 So I propose to get rid of saving keypair in nailgun database and
 implement a way to always saving it to local FS on master node. We need to
 implement next items:

 - Change UI logic that saving keypair into DB to logic that will save it
 to local FS
 - Implement according fixes in fuel-library

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] SSL for master node API

2015-08-04 Thread Evgeniy L

Hi,

+1 to 2nd solution, in this case old environments will work without
additional
actions. Agents for new environments, CLI and UI will use SSL.
But probably for UI we will have to perform redirect on JS level.

Thanks,

On Tue, Aug 4, 2015 at 1:32 PM, Stanislaw Bogatkin sbogat...@mirantis.com
wrote:

 Hi guys,
 in overall movement of Fuel to use secure sockets we think about wrapping
 master node UI and API calls to SSL. But there are next caveat:

 a) fuel-nailgun-agent cannot work via SSL now and need to be rewritten a
 little. But if it will be rewritten in 7.0 and HTTPS on master node will be
 forced by default, it will break upgrade from previous releases to 7.0 due
 fact that after master node upgrade from 6.1 to 7.0 we will have HTTPS by
 default and fuel-nailgun-agent on all environments won't upgraded, so it
 won't be able to connect to master node after upgrade. It breaks seamless
 upgrade procedure.

 What options I see there:
 1. We can forcedly enable SSL for master node and rewrite clients in 7.0
 to be able to work over it. In release notes for 7.0 we will write
 forewarning that clients which want to upgrade master node from previous
 releases to 7.0 must also install new fuel-nailgun-agent to all nodes in
 all deployed environments.

 2. We can have both SSL and non-SSL versions enabled by default and
 rewrite fuel-nailgun-client in 7.0 such way that it will check SSL
 availability and be able to work in plain HTTP for legacy mode. So, for all
 new environments SSL will be used by default and for old ones plain HTTP
 will continue to work too. Master node upgrade will not be broken in this
 case.

 3. We can do some mixed way by gradually rewrite fuel-nailgun-client, save
 both HTTP and HTTPS for master node in 7.0 and drop plain HTTP in next
 releases. It is just postponed version of first clause, so it doesn't seems
 valid for me, actually.

 I would be really glad to hear what you think about this. Thank you in
 advance.

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins] Feedback

2015-07-30 Thread Evgeniy L

Hi Sheena,

Created ticket to change the structure of the directories [1].
And as far as I know any core can push tags into the repository,
Sebastian, Igor and I.

[1] https://bugs.launchpad.net/fuel/+bug/1479785

On Tue, Jul 28, 2015 at 8:44 PM, Sheena Gregson sgreg...@mirantis.com
wrote:

Evgeniy –

For the items which you have listed actions, who should be responsible for
next steps?

Sheena

*From:* Evgeniy L [mailto:e...@mirantis.com]
*Sent:* Tuesday, July 28, 2015 11:54 AM
*To:* OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
*Subject:* Re: [openstack-dev] [Fuel][Plugins] Feedback

Hi Sergii, thank you for feedback,

c. There is no documentation how to install fpb from github master
branch. It's very useful for developers who want to use latest version. We
should add something

We had a documentation, but removed it because the newer fpb was released,

probably we should add this information permanently [1].

a. We are doing the same mistake putting all things into one basket.
There should be 2 repositories. One for examples and one for fpb. What's
the goal of keeping fpb in directory and examples on top?

These plugins are the data which are required for integration testing,

we test that plugin build is not broken, which we run when patch gets

published. I see nothing wrong with having the data for integration testing

in the same repository with product which should be tested.

Also in previous release we *removed* all the plugins which are not

related to the builder itself, lbaas and glusterfs.

This breaks a couple of things

Having data for testing in the repository doesn't break anything.

b. I cannot build fpm with simple

That is a good point, we should move code from fuel_plugin_builder
directory

on top level, and move data for testing into examples directory.

c. There is no tags as I can see only stable/6.0

Correct, tags should be added.

d. There are no tests to improve code quality pep8 flask8, code coverage

That is not true, there are more then one hundreds unit tests which we run

for each patch with python 2.6 and python 2.7, also there are integration
tests

which check that for each patch we don't break validation and that we can

build plugins for previous versions. Plus there are functional tests which
are

written by fuel-qa team, those tests check that we perform deployment

with plugins and required functionality works correctly. Also there *is*
pep8

check [2].

e. Repository doesn't follow community standards.

I think this issue should be resolved with moving fuel_plugin_builder
directory

on level higher, if not, please provide more specific description what is
wrong.

3. Setting tab ...

Agree.

[1]
https://wiki.openstack.org/w/index.php?title=Fuel%2FPluginsdiff=78677oldid=78204

[2]
https://github.com/stackforge/fuel-plugins/blob/master/fuel_plugin_builder/tox.ini#L17-L21

On Tue, Jul 28, 2015 at 5:51 PM, Sergii Golovatiuk
sgolovat...@mirantis.com wrote:

Hi,

I have started digging into plugins recently. There are many positive
things though I would like to point to some problem areas

1. Documentation

a. It doesn't include the features of 7.0. There are many outstanding
features, though I needed to ping the developers to ask how these features
work. It means that it's almost impossible to develop plugins for upcoming
releases. The external developer needs to wait for documentation so it
creates a lag between release and plugin release.

b. in [1] the statement about 'For Ubuntu 12.04.2 LTS' should be extended
to 14.04. Also we don't need to add PATCH version as 12.04.2 is equivalent
to 12.04

c. There is no documentation how to install fpb from github master branch.
It's very useful for developers who want to use latest version. We should
add something

2. Github repository [2] is messed up

a. We are doing the same mistake putting all things into one basket. There
should be 2 repositories. One for examples and one for fpb. What's the goal
of keeping fpb in directory and examples on top? This breaks a couple of
things

b. I cannot build fpm with simple

pip install git+https://

Instead I am forced to do

git clone https://

cd fuel-plugins

pip install .

c. There is no tags as I can see only stable/6.0

d. There are no tests to improve code quality pep8 flask8, code coverage

e. Repository doesn't follow community standards.

3. Setting tab

When plugin is installed, it's very hard to find in. In setting tab it's
somewhere between A and Z

How is user supposed to find it? There should be a separator between Core
features and plugins. User must easily find, configure, enable/disable them.

P.S. I am asking everyone to add own concerns so we'll be able to make a
plan how to address them.

Thank you in advance.

[1] https://wiki.openstack.org/wiki/Fuel/Plugins

Re: [openstack-dev] [fuel] Fuel plugin as docker containers images

Hi Konstantin,

I'm not sure if we track such feature anywhere. But one of the reasons
to use containers on the master was to deliver plugin specific containers,
so they don't intersect and don't break Fuel master dependencies.
Do you have some specific use case for that?

Thanks,

On Mon, Jul 27, 2015 at 4:58 PM, Konstantin Danilov kdani...@mirantis.com
wrote:

 Hi,

 Is there a plans to allow plugin to be delivered as docker container
 images?

 Thanks

 --
 Kostiantyn Danilov aka koder.ua
 Principal software engineer, Mirantis

 skype:koder.ua
 http://koder-ua.blogspot.com/
 http://mirantis.com

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] FF Exception request for Templates for Networking feature

Hi Alexander, I don't agree with your statements

[1] - I just uses % and % to substitute values.

It's what templating is about, you have some text preprocessor to
substitute values.

That is not ERB style template language.

ERB uses the same syntax, hence it Is ERB style.

[2] - We are not using Jinja templating (it is just yaml file, not html
template), we are using Jinja placeholder substitution.

We *are using* jinja templating (I don't understand why you mention here
html),
with all it's features and here is the proof [1].

And in current code we have a problem with content at first parsed from
yaml and that parser treats { and [ as a start if a dict or an array.

key: {{blha}}

[3] - Templates are for people who do not care about Jinja/ERB (maybe
some familiar with Puppet/Chef), so no confusion.

That is not correct, every template has it's own syntax, so people have to
care about
specific implementation i.e. Jinja or ERB, and there will be confusion when
somebody will
try to use ERB specific features, and she/he will fail because you hide
Jinja under ERB
syntax.

[1]
https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/objects/node.py#L854-L855

On Tue, Jul 28, 2015 at 11:29 AM, Alexander Kostrikov
akostri...@mirantis.com wrote:

Completely agree with Sergey.

Currently network template uses ERB [1] style template language,
but in fact it's Jinja [2], it was agreed to change it [3], no to confuse
the user

[1] - I just uses % and % to substitute values. That is not ERB style
template language.
[2] - We are not using Jinja templating (it is just yaml file, not html
template), we are using Jinja placeholder substitution. And in current code
we have a problem with content at first parsed from yaml and that parser
treats { and [ as a start if a dict or an array.
[3] - Templates are for people who do not care about Jinja/ERB (maybe some
familiar with Puppet/Chef), so no confusion.

On Tue, Jul 28, 2015 at 10:57 AM, Sergey Vasilenko
svasile...@mirantis.com wrote:

On Mon, Jul 27, 2015 at 1:10 PM, Evgeniy L e...@mirantis.com wrote:

Currently network template uses ERB [1] style template language,
but in fact it's Jinja [2], it was agreed to change it [3], no to confuse
the user, with ERB which is in fact jinja and doesn't have any ERB
features.

we have not so much syntax choices for convenient templating. Network
temptales will be used by system administrators.
The '% %' pair is a de-facto standart in this area. In the puppet
world. Not every system administrator will meaning ERB when seeing '% %'
pair.
Another pairs (i.e. '$ $' , ' ', etc) looks more
non-standart. Plenty of syntax features are annoy and make usability of
product less convenient.
I propose do not make extra essences on the clean area...

We never say in the docs about ERB. IMHO it's enough for leave '% %'
as is.

/sv

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Kind Regards,

Alexandr Kostrikov,

Mirantis, Inc.

35b/3, Vorontsovskaya St., 109147, Moscow, Russia

Tel.: +7 (495) 640-49-04
Tel.: +7 (925) 716-64-52 %2B7%20%28906%29%20740-64-79

Skype: akostrikov_mirantis

E-mail: akostri...@mirantis.com elogut...@mirantis.com

*www.mirantis.com http://www.mirantis.ru/*
*www.mirantis.ru http://www.mirantis.ru/*

Re: [openstack-dev] [fuel] FF Exception request for Templates for Networking feature

Hi Sergey,

Thanks, now I see why we had misunderstanding. The problem is currently
all set of features which Jinja2 provides is available for the user. As far
as
I know there is no way in Jinja to disable all of the functionality except
just
substitution.

If we need only substitution, probably it's better to use standard
templating
in python [1], there is a way to redefine tokens, so you will be able to use
% % syntax if you want to.

[1] https://docs.python.org/2.6/library/string.html#template-strings
https://docs.python.org/dev/library/string.html#template-strings
[2] http://pymotw.com/2/string/#advanced-templates

On Tue, Jul 28, 2015 at 12:25 PM, Sergey Vasilenko svasile...@mirantis.com
wrote:


 On Tue, Jul 28, 2015 at 11:52 AM, Evgeniy L e...@mirantis.com wrote:

 Hi Alexander, I don't agree with your statements

  [1] - I just uses % and % to substitute values.

 It's what templating is about, you have some text preprocessor to
 substitute values.


 Network templates feature don't mean any  text preprocessor actions.
 Only value substitutions


  That is not ERB style template language.

 ERB uses the same syntax, hence it Is ERB style.


 ... hence it looks like ERB. not more.
 Not only ruby used for programming. Non only EBD -- is template language.
 ;)


  [2] - We are not using Jinja templating (it is just yaml file, not
 html template), we are using Jinja placeholder substitution.

 We *are using* jinja templating (I don't understand why you mention here
 html),
 with all it's features and here is the proof [1].


 We don't promise use Junja (or whatever) template language for this
 feature.
 If some jinja features allowed for parsing Network template -- it's a bug.
 We should check it and fix it.
 Only value substitutions should allow in the network templates.


  [3] - Templates are for people who do not care about Jinja/ERB (maybe
 some familiar with Puppet/Chef), so no confusion.

 That is not correct, every template has it's own syntax, so people have
 to care about
 specific implementation i.e. Jinja or ERB, and there will be confusion
 when somebody will
 try to use ERB specific features, and she/he will fail because you hide
 Jinja under ERB
 syntax.


 I, partially, agree with you. But please honored to following facts:
 * In the deployers world used Jinja and ERB syntax.
 * ERB used more often, because Ansible (I don't know another popular
 deployers tools with Jinja templating) is to young.
 * Plenty of syntax features is a really hell.

 In the Network templates we don't suppose anything other than a simple
 substitution variable values.
 All logic of template processing implementing on Nailgun side. Now on the
 template parsing, later -- on the network manipulation class.
 Allowance of mix template language and Nailgun logic may lead to heavy
 diagnostic issues.
 Meantime I don't see any cases, where required something more, than
 substitution.



 /sv

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] FF Exception request for Templates for Networking feature

Aleksey, here is working version [1].

 Evgeniy, do we need to remove jinja before July 30th ?

With this issue feature can leave, and it won't have huge user impact.
At the same time by design we didn't want to have anything except
substitution, hence it's probably as Sergey mentioned is a bug.
So looks like it can be fixed after 30th of July, and should not be
considered as a blocker. Validation is much more important, so lets
fix it first.

[1] http://paste.openstack.org/show/406104/


On Tue, Jul 28, 2015 at 7:23 PM, Sergey Vasilenko svasile...@mirantis.com
wrote:


 On Tue, Jul 28, 2015 at 7:13 PM, Aleksey Kasatkin akasat...@mirantis.com
 wrote:

 Evgeniy, do we need to remove jinja before July 30th ?


 I think -- not.
 It just a bug, not a key-point of feature.


 /sv

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Plugins] Feedback