Re: [openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-29 Thread Dmitry Borodaenko 
On Mon, Feb 29, 2016 at 08:21:18AM +0100, Andreas Jaeger wrote:
> > Yes, they will officially become a part of Fuel project. First such
> > example is likely to be the Murano plugin [0], so we can use it a the
> > guinea pig to try out this process.
> > 
> > [0] https://review.openstack.org/269567
> > 
> > As described in the corresponding spec [1], we plan this plugin to
> > co-exist with the current non-plugin implementation for Mitaka, and then
> > let the legacy non-plugin implementation be superceded by
> > fuel-plugin-murano as soon as the latter reaches maturity (hopefully
> > early in Newton cycle).
> > 
> > [1] https://review.openstack.org/275124
> 
> Why is the corresponding spec not mentioned as part of the infra change?

We updated the change to reference the blueprint which in turn links to
this spec.

> > Since this is not a straight-forward moving of existing code into its
> > own git repo (like we've recently done for fuel-virtualbox and fuel-ui),
> > I think it's too early to register the new plugin repo in
> > openstack/governance, but eventually (as soon as it's ready to become
> > the default way to deploy Murano with Fuel) we should add it there.
> 
> That's not a reason under the big tent to not add it to the governance
> documentation. There's no need to incubate it outside the tree.

Fair enough. I've posted a governance change and linked it to the infra
change:

https://review.openstack.org/286310

-- 
Dmitry Borodaenko

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-28 Thread Andreas Jaeger 
On 2016-02-27 00:54, Dmitry Borodaenko wrote:
> On Thu, Feb 25, 2016 at 08:31:26AM +0100, Andreas Jaeger wrote:
>> On 2016-02-03 03:27, Dmitry Borodaenko wrote:
>>> [...]
>>> Level 1. Plugin is actively supported by Fuel team
>>>
>>> As we expand plugin capabilities and move more functionality from Fuel
>>> core into plugins, we will inevitably get to the point where some
>>> plugins are required to successfully complete even a basic deployment
>>> (aka "pass BVT"). Even before that, we may decide that some plugins are
>>> important enough for our reference architecture to allow the state of
>>> these plugins to affect our release cycle: allow Critical bugs in them
>>> to affect Fuel release, cover them in acceptance testing for Fuel
>>> releases and maintenance updates, and so on.
>>
>>
>>> Obviously, whole Fuel team is expected to support such plugins, and is
>>> self-motivated to provide timely help to their maintainers to keep them
>>> healthy. In addition to the expectations from the previous support
>>> level, we should track the list of these release-critical plugins in the
>>> policy section of fuel-specs [7].
>>
>> So, these are plugins that will be officially part of fuel team? You
>> depend on it for successful installation...
> 
> Yes, they will officially become a part of Fuel project. First such
> example is likely to be the Murano plugin [0], so we can use it a the
> guinea pig to try out this process.
> 
> [0] https://review.openstack.org/269567
> 
> As described in the corresponding spec [1], we plan this plugin to
> co-exist with the current non-plugin implementation for Mitaka, and then
> let the legacy non-plugin implementation be superceded by
> fuel-plugin-murano as soon as the latter reaches maturity (hopefully
> early in Newton cycle).
> 
> [1] https://review.openstack.org/275124

Why is the corresponding spec not mentioned as part of the infra change?

> 
> Since this is not a straight-forward moving of existing code into its
> own git repo (like we've recently done for fuel-virtualbox and fuel-ui),
> I think it's too early to register the new plugin repo in
> openstack/governance, but eventually (as soon as it's ready to become
> the default way to deploy Murano with Fuel) we should add it there.

That's not a reason under the big tent to not add it to the governance
documentation. There's no need to incubate it outside the tree.

Andreas
-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-26 Thread Dmitry Borodaenko 
Excellent point, and also a good reminder that we should fix our system
tests documentation, at the moment it's slightly too short:

https://docs.fuel-infra.org/fuel-dev/develop/system_tests/tree.html

The most relevant docuement I could find was this spec:

https://specs.openstack.org/openstack/fuel-specs/specs/8.0/template-based-virtual-devops-environments.html

On Tue, Feb 16, 2016 at 12:45:51PM +0300, Evgeniy L wrote:
> Hi,
> 
> I have some comments on CI for plugins, currently there is a good
> instruction on how to install your own CI and using fuel-qa write your own
> tests [0], since just running BVT is not enough to make sure that plugin
> works, we should provide a way for a plugin developer to easily extend
> checks/asserts which are done during BVT by just putting python files into
> plugin-name/tests directory. This way will be able to install plugin
> (enable it) and perform plugin specific checks.
> 
> Thanks,
> 
> [0] https://wiki.openstack.org/wiki/Fuel/Plugins#CI
> 
> On Wed, Feb 3, 2016 at 5:27 AM, Dmitry Borodaenko 
> wrote:
> 
> > It has been over a year since pluggable architecture was introduced in
> > Fuel 6.0, and I think it's safe to declare it an unmitigated success. A
> > search for "fuel-plugin" on GitHub brings up 167 repositories [0],
> > there's 63 Fuel plugin repositories on review.openstack.org [1], 25 Fuel
> > plugins are listed in the DriverLog [2].
> >
> > [0] https://github.com/search?q=fuel-plugin-
> > [1]
> > https://review.openstack.org/#/admin/projects/?filter=openstack%252Ffuel-plugin-
> > [2] http://stackalytics.com/report/driverlog?project_id=openstack%2Ffuel
> >
> > Even though the plugin engine is not yet complete (there still are
> > things you can do in Fuel core that you cannot do in a plugin), dozens
> > of deployers and developers [3] used it to expand Fuel capabilities
> > beyond the limitations of our default reference architecture.
> >
> > [3] http://stackalytics.com/report/contribution/fuel-plugins-group/360
> >
> > There's a noticeable bump in contributions around October 2015 after
> > Fuel 7.0 was released, most likely inspired by the plugin engine
> > improvements introduced in that version [4]. As we continue to expand
> > plugins capabilities, I expect more and more plugins to appear.
> >
> > [4]
> > https://git.openstack.org/cgit/openstack/fuel-docs/tree/pages/release-notes/v7-0/new_features/plugins.rst?h=stable/7.0
> >
> > The question of how useful exactly all those plugins are is a bit harder
> > to answer. DriverLog isn't much help: less than half of Fuel plugins
> > hosted on OpenStack infrastructure are even registered there, and of
> > those that are, only 6 have CI jobs with recent successful runs. Does
> > this mean that 90% of Fuel plugins are broken and unmaintained? Not
> > necessarily, but it does mean that we have no way to tell.
> >
> > An even harder question is: once we determine that some plugins are more
> > equal than others, what should we do about the less useful and the less
> > actively maintained?
> >
> > To objectively answer both questions, we need to define support levels
> > for Fuel plugins and set some reasonable expectations about how plugins
> > can qualify for each level.
> >
> > Level 3. Plugin is not actively supported
> >
> > I believe that having hundreds of Fuel plugins out there on GitHub and
> > elsewhere is great, and we should encourage people to create more of
> > those and do whatever they like with them. Even a single-commit "deploy
> > and forget" plugin is useful as an idea, a source of inspiration, and a
> > starting point for other people who might want to take it further.
> >
> > At this level, there should be zero expectations and zero obligations
> > between Fuel plugin writers and OpenStack community. At the moment, Fuel
> > plugins developers guide recommends [5] to request a Gerrit repo in the
> > openstack/ namespace and set up branches, tags, CI, and a code review
> > process around it, aligned with OpenStack development process. Which is
> > generally a good idea, except for all the cases where it's too much
> > overhead and ends up not being followed closely enough to be useful.
> >
> > [5] https://wiki.openstack.org/wiki/Fuel/Plugins#Repo
> >
> > Instead of vague blanket recommendations, we should explictly state that
> > it's fine to do none of that and just stay on GitHub, and that if you
> > intend to move to the next level and actively maintain your plugin, and
> > expect support with that from Fuel developers and other OpenStack
> > projects, these recommendations are not optional and must be fulfilled.
> >
> > Level 2. Plugin is actively supported by its registered maintainers
> >
> > To support a Fuel plugin, we need to answer two fundamental questions:
> > Can we? Should we?
> >
> > I think the minimum requirements to say "yes" to both are:
> >
> > a) All of the plugin's source code is explicitly licensed under an
> >OSI-approved license;
> >
> >

Re: [openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-26 Thread Dmitry Borodaenko 
On Thu, Feb 25, 2016 at 08:31:26AM +0100, Andreas Jaeger wrote:
> On 2016-02-03 03:27, Dmitry Borodaenko wrote:
> > [...]
> > Level 1. Plugin is actively supported by Fuel team
> > 
> > As we expand plugin capabilities and move more functionality from Fuel
> > core into plugins, we will inevitably get to the point where some
> > plugins are required to successfully complete even a basic deployment
> > (aka "pass BVT"). Even before that, we may decide that some plugins are
> > important enough for our reference architecture to allow the state of
> > these plugins to affect our release cycle: allow Critical bugs in them
> > to affect Fuel release, cover them in acceptance testing for Fuel
> > releases and maintenance updates, and so on.
> 
> 
> > Obviously, whole Fuel team is expected to support such plugins, and is
> > self-motivated to provide timely help to their maintainers to keep them
> > healthy. In addition to the expectations from the previous support
> > level, we should track the list of these release-critical plugins in the
> > policy section of fuel-specs [7].
> 
> So, these are plugins that will be officially part of fuel team? You
> depend on it for successful installation...

Yes, they will officially become a part of Fuel project. First such
example is likely to be the Murano plugin [0], so we can use it a the
guinea pig to try out this process.

[0] https://review.openstack.org/269567

As described in the corresponding spec [1], we plan this plugin to
co-exist with the current non-plugin implementation for Mitaka, and then
let the legacy non-plugin implementation be superceded by
fuel-plugin-murano as soon as the latter reaches maturity (hopefully
early in Newton cycle).

[1] https://review.openstack.org/275124

Since this is not a straight-forward moving of existing code into its
own git repo (like we've recently done for fuel-virtualbox and fuel-ui),
I think it's too early to register the new plugin repo in
openstack/governance, but eventually (as soon as it's ready to become
the default way to deploy Murano with Fuel) we should add it there.

> Andreas
> 
> 
> > [7] https://git.openstack.org/cgit/openstack/fuel-specs/tree/policy
> > 
> > Thoughts, comments, objections?
> > 
> 
> 
> -- 
>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-26 Thread Dmitry Borodaenko 
Yes, this is a reasonable way to handle repo creation requests, and we
should update our wiki to reflect this:
https://wiki.openstack.org/wiki/Fuel/Plugins#Repo

On Mon, Feb 15, 2016 at 05:01:39PM +0100, Mateusz Matuszkowiak wrote:
> Dmitry,
> 
> So this changes the workflow for the devopses, the fuel plugin repo creators 
> under Openstack namespace.
> 
> As I understand, development of every new fuel plugin must be now started in 
> a private github repo first, 

Not necessarily github, any public git hosting should work just as well.

> and when a developer(s) decide they want to go level higher they request for 
> a validation. 
> And only after successfull validation they should request a repository 
> creation (via LP bug) for their plugin under Openstack NS, right? 
> I’m asking since its important for us to properly handle these kind of bugs 
> [0].
> 
> Regards,
> 
> [0] https://bugs.launchpad.net/fuel/+bug/1544536 
> 
> 
> --
> Fuel DevOps
> Mateusz Matuszkowiak
> 
> 
> > On Feb 3, 2016, at 3:27 AM, Dmitry Borodaenko  
> > wrote:
> > 
> > It has been over a year since pluggable architecture was introduced in
> > Fuel 6.0, and I think it's safe to declare it an unmitigated success. A
> > search for "fuel-plugin" on GitHub brings up 167 repositories [0],
> > there's 63 Fuel plugin repositories on review.openstack.org [1], 25 Fuel
> > plugins are listed in the DriverLog [2].
> > 
> > [0] https://github.com/search?q=fuel-plugin-
> > [1] 
> > https://review.openstack.org/#/admin/projects/?filter=openstack%252Ffuel-plugin-
> > [2] http://stackalytics.com/report/driverlog?project_id=openstack%2Ffuel
> > 
> > Even though the plugin engine is not yet complete (there still are
> > things you can do in Fuel core that you cannot do in a plugin), dozens
> > of deployers and developers [3] used it to expand Fuel capabilities
> > beyond the limitations of our default reference architecture.
> > 
> > [3] http://stackalytics.com/report/contribution/fuel-plugins-group/360
> > 
> > There's a noticeable bump in contributions around October 2015 after
> > Fuel 7.0 was released, most likely inspired by the plugin engine
> > improvements introduced in that version [4]. As we continue to expand
> > plugins capabilities, I expect more and more plugins to appear.
> > 
> > [4] 
> > https://git.openstack.org/cgit/openstack/fuel-docs/tree/pages/release-notes/v7-0/new_features/plugins.rst?h=stable/7.0
> > 
> > The question of how useful exactly all those plugins are is a bit harder
> > to answer. DriverLog isn't much help: less than half of Fuel plugins
> > hosted on OpenStack infrastructure are even registered there, and of
> > those that are, only 6 have CI jobs with recent successful runs. Does
> > this mean that 90% of Fuel plugins are broken and unmaintained? Not
> > necessarily, but it does mean that we have no way to tell.
> > 
> > An even harder question is: once we determine that some plugins are more
> > equal than others, what should we do about the less useful and the less
> > actively maintained?
> > 
> > To objectively answer both questions, we need to define support levels
> > for Fuel plugins and set some reasonable expectations about how plugins
> > can qualify for each level.
> > 
> > Level 3. Plugin is not actively supported
> > 
> > I believe that having hundreds of Fuel plugins out there on GitHub and
> > elsewhere is great, and we should encourage people to create more of
> > those and do whatever they like with them. Even a single-commit "deploy
> > and forget" plugin is useful as an idea, a source of inspiration, and a
> > starting point for other people who might want to take it further.
> > 
> > At this level, there should be zero expectations and zero obligations
> > between Fuel plugin writers and OpenStack community. At the moment, Fuel
> > plugins developers guide recommends [5] to request a Gerrit repo in the
> > openstack/ namespace and set up branches, tags, CI, and a code review
> > process around it, aligned with OpenStack development process. Which is
> > generally a good idea, except for all the cases where it's too much
> > overhead and ends up not being followed closely enough to be useful.
> > 
> > [5] https://wiki.openstack.org/wiki/Fuel/Plugins#Repo
> > 
> > Instead of vague blanket recommendations, we should explictly state that
> > it's fine to do none of that and just stay on GitHub, and that if you
> > intend to move to the next level and actively maintain your plugin, and
> > expect support with that from Fuel developers and other OpenStack
> > projects, these recommendations are not optional and must be fulfilled.
> > 
> > Level 2. Plugin is actively supported by its registered maintainers 
> > 
> > To support a Fuel plugin, we need to answer two fundamental questions:
> > Can we? Should we?
> > 
> > I think the minimum requirements to say "yes" to both are:
> > 
> > a) All of the plugin's source code is

Re: [openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-24 Thread Andreas Jaeger 
On 2016-02-03 03:27, Dmitry Borodaenko wrote:
> [...]
> Level 1. Plugin is actively supported by Fuel team
> 
> As we expand plugin capabilities and move more functionality from Fuel
> core into plugins, we will inevitably get to the point where some
> plugins are required to successfully complete even a basic deployment
> (aka "pass BVT"). Even before that, we may decide that some plugins are
> important enough for our reference architecture to allow the state of
> these plugins to affect our release cycle: allow Critical bugs in them
> to affect Fuel release, cover them in acceptance testing for Fuel
> releases and maintenance updates, and so on.


> Obviously, whole Fuel team is expected to support such plugins, and is
> self-motivated to provide timely help to their maintainers to keep them
> healthy. In addition to the expectations from the previous support
> level, we should track the list of these release-critical plugins in the
> policy section of fuel-specs [7].

So, these are plugins that will be officially part of fuel team? You
depend on it for successful installation...

Andreas


> [7] https://git.openstack.org/cgit/openstack/fuel-specs/tree/policy
> 
> Thoughts, comments, objections?
> 


-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-16 Thread Evgeniy L 
Hi,

I have some comments on CI for plugins, currently there is a good
instruction on how to install your own CI and using fuel-qa write your own
tests [0], since just running BVT is not enough to make sure that plugin
works, we should provide a way for a plugin developer to easily extend
checks/asserts which are done during BVT by just putting python files into
plugin-name/tests directory. This way will be able to install plugin
(enable it) and perform plugin specific checks.

Thanks,

[0] https://wiki.openstack.org/wiki/Fuel/Plugins#CI

On Wed, Feb 3, 2016 at 5:27 AM, Dmitry Borodaenko 
wrote:

> It has been over a year since pluggable architecture was introduced in
> Fuel 6.0, and I think it's safe to declare it an unmitigated success. A
> search for "fuel-plugin" on GitHub brings up 167 repositories [0],
> there's 63 Fuel plugin repositories on review.openstack.org [1], 25 Fuel
> plugins are listed in the DriverLog [2].
>
> [0] https://github.com/search?q=fuel-plugin-
> [1]
> https://review.openstack.org/#/admin/projects/?filter=openstack%252Ffuel-plugin-
> [2] http://stackalytics.com/report/driverlog?project_id=openstack%2Ffuel
>
> Even though the plugin engine is not yet complete (there still are
> things you can do in Fuel core that you cannot do in a plugin), dozens
> of deployers and developers [3] used it to expand Fuel capabilities
> beyond the limitations of our default reference architecture.
>
> [3] http://stackalytics.com/report/contribution/fuel-plugins-group/360
>
> There's a noticeable bump in contributions around October 2015 after
> Fuel 7.0 was released, most likely inspired by the plugin engine
> improvements introduced in that version [4]. As we continue to expand
> plugins capabilities, I expect more and more plugins to appear.
>
> [4]
> https://git.openstack.org/cgit/openstack/fuel-docs/tree/pages/release-notes/v7-0/new_features/plugins.rst?h=stable/7.0
>
> The question of how useful exactly all those plugins are is a bit harder
> to answer. DriverLog isn't much help: less than half of Fuel plugins
> hosted on OpenStack infrastructure are even registered there, and of
> those that are, only 6 have CI jobs with recent successful runs. Does
> this mean that 90% of Fuel plugins are broken and unmaintained? Not
> necessarily, but it does mean that we have no way to tell.
>
> An even harder question is: once we determine that some plugins are more
> equal than others, what should we do about the less useful and the less
> actively maintained?
>
> To objectively answer both questions, we need to define support levels
> for Fuel plugins and set some reasonable expectations about how plugins
> can qualify for each level.
>
> Level 3. Plugin is not actively supported
>
> I believe that having hundreds of Fuel plugins out there on GitHub and
> elsewhere is great, and we should encourage people to create more of
> those and do whatever they like with them. Even a single-commit "deploy
> and forget" plugin is useful as an idea, a source of inspiration, and a
> starting point for other people who might want to take it further.
>
> At this level, there should be zero expectations and zero obligations
> between Fuel plugin writers and OpenStack community. At the moment, Fuel
> plugins developers guide recommends [5] to request a Gerrit repo in the
> openstack/ namespace and set up branches, tags, CI, and a code review
> process around it, aligned with OpenStack development process. Which is
> generally a good idea, except for all the cases where it's too much
> overhead and ends up not being followed closely enough to be useful.
>
> [5] https://wiki.openstack.org/wiki/Fuel/Plugins#Repo
>
> Instead of vague blanket recommendations, we should explictly state that
> it's fine to do none of that and just stay on GitHub, and that if you
> intend to move to the next level and actively maintain your plugin, and
> expect support with that from Fuel developers and other OpenStack
> projects, these recommendations are not optional and must be fulfilled.
>
> Level 2. Plugin is actively supported by its registered maintainers
>
> To support a Fuel plugin, we need to answer two fundamental questions:
> Can we? Should we?
>
> I think the minimum requirements to say "yes" to both are:
>
> a) All of the plugin's source code is explicitly licensed under an
>OSI-approved license;
>
> b) The plugin source code repository does not contain binary artefacts
>such as RPM packages or ISO images (*);
>
> c) The plugin is registered in DriverLog;
>
> d) Plugin maintainers listed in DriverLog have confirmed the intent to
>support the plugin;
>
> e) Plugin repository on review.openstack.org has a voting CI job that is
>passing with the latest or, at least, previous major release of Fuel.
>
> f) All deviations from the OpenStack development process (alternative
>issue trackers, mailing lists, etc.) are documented in the plugin's
>README file.
>
> *  Aside from purely

Re: [openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-15 Thread Mateusz Matuszkowiak 
Dmitry,

So this changes the workflow for the devopses, the fuel plugin repo creators 
under Openstack namespace.

As I understand, development of every new fuel plugin must be now started in a 
private github repo first, 
and when a developer(s) decide they want to go level higher they request for a 
validation. 
And only after successfull validation they should request a repository creation 
(via LP bug) for their plugin under Openstack NS, right? 
I’m asking since its important for us to properly handle these kind of bugs [0].

Regards,

[0] https://bugs.launchpad.net/fuel/+bug/1544536 


--
Fuel DevOps
Mateusz Matuszkowiak


> On Feb 3, 2016, at 3:27 AM, Dmitry Borodaenko  
> wrote:
> 
> It has been over a year since pluggable architecture was introduced in
> Fuel 6.0, and I think it's safe to declare it an unmitigated success. A
> search for "fuel-plugin" on GitHub brings up 167 repositories [0],
> there's 63 Fuel plugin repositories on review.openstack.org [1], 25 Fuel
> plugins are listed in the DriverLog [2].
> 
> [0] https://github.com/search?q=fuel-plugin-
> [1] 
> https://review.openstack.org/#/admin/projects/?filter=openstack%252Ffuel-plugin-
> [2] http://stackalytics.com/report/driverlog?project_id=openstack%2Ffuel
> 
> Even though the plugin engine is not yet complete (there still are
> things you can do in Fuel core that you cannot do in a plugin), dozens
> of deployers and developers [3] used it to expand Fuel capabilities
> beyond the limitations of our default reference architecture.
> 
> [3] http://stackalytics.com/report/contribution/fuel-plugins-group/360
> 
> There's a noticeable bump in contributions around October 2015 after
> Fuel 7.0 was released, most likely inspired by the plugin engine
> improvements introduced in that version [4]. As we continue to expand
> plugins capabilities, I expect more and more plugins to appear.
> 
> [4] 
> https://git.openstack.org/cgit/openstack/fuel-docs/tree/pages/release-notes/v7-0/new_features/plugins.rst?h=stable/7.0
> 
> The question of how useful exactly all those plugins are is a bit harder
> to answer. DriverLog isn't much help: less than half of Fuel plugins
> hosted on OpenStack infrastructure are even registered there, and of
> those that are, only 6 have CI jobs with recent successful runs. Does
> this mean that 90% of Fuel plugins are broken and unmaintained? Not
> necessarily, but it does mean that we have no way to tell.
> 
> An even harder question is: once we determine that some plugins are more
> equal than others, what should we do about the less useful and the less
> actively maintained?
> 
> To objectively answer both questions, we need to define support levels
> for Fuel plugins and set some reasonable expectations about how plugins
> can qualify for each level.
> 
> Level 3. Plugin is not actively supported
> 
> I believe that having hundreds of Fuel plugins out there on GitHub and
> elsewhere is great, and we should encourage people to create more of
> those and do whatever they like with them. Even a single-commit "deploy
> and forget" plugin is useful as an idea, a source of inspiration, and a
> starting point for other people who might want to take it further.
> 
> At this level, there should be zero expectations and zero obligations
> between Fuel plugin writers and OpenStack community. At the moment, Fuel
> plugins developers guide recommends [5] to request a Gerrit repo in the
> openstack/ namespace and set up branches, tags, CI, and a code review
> process around it, aligned with OpenStack development process. Which is
> generally a good idea, except for all the cases where it's too much
> overhead and ends up not being followed closely enough to be useful.
> 
> [5] https://wiki.openstack.org/wiki/Fuel/Plugins#Repo
> 
> Instead of vague blanket recommendations, we should explictly state that
> it's fine to do none of that and just stay on GitHub, and that if you
> intend to move to the next level and actively maintain your plugin, and
> expect support with that from Fuel developers and other OpenStack
> projects, these recommendations are not optional and must be fulfilled.
> 
> Level 2. Plugin is actively supported by its registered maintainers 
> 
> To support a Fuel plugin, we need to answer two fundamental questions:
> Can we? Should we?
> 
> I think the minimum requirements to say "yes" to both are:
> 
> a) All of the plugin's source code is explicitly licensed under an
>   OSI-approved license;
> 
> b) The plugin source code repository does not contain binary artefacts
>   such as RPM packages or ISO images (*);
> 
> c) The plugin is registered in DriverLog;
> 
> d) Plugin maintainers listed in DriverLog have confirmed the intent to
>   support the plugin;
> 
> e) Plugin repository on review.openstack.org has a voting CI job that is
>   passing with the latest or, at least, previous major release of Fuel.
> 
> f) All deviations from the

[openstack-dev] [fuel] Fuel plugins: lets have some rules

2016-02-02 Thread Dmitry Borodaenko 
It has been over a year since pluggable architecture was introduced in
Fuel 6.0, and I think it's safe to declare it an unmitigated success. A
search for "fuel-plugin" on GitHub brings up 167 repositories [0],
there's 63 Fuel plugin repositories on review.openstack.org [1], 25 Fuel
plugins are listed in the DriverLog [2].

[0] https://github.com/search?q=fuel-plugin-
[1] 
https://review.openstack.org/#/admin/projects/?filter=openstack%252Ffuel-plugin-
[2] http://stackalytics.com/report/driverlog?project_id=openstack%2Ffuel

Even though the plugin engine is not yet complete (there still are
things you can do in Fuel core that you cannot do in a plugin), dozens
of deployers and developers [3] used it to expand Fuel capabilities
beyond the limitations of our default reference architecture.

[3] http://stackalytics.com/report/contribution/fuel-plugins-group/360

There's a noticeable bump in contributions around October 2015 after
Fuel 7.0 was released, most likely inspired by the plugin engine
improvements introduced in that version [4]. As we continue to expand
plugins capabilities, I expect more and more plugins to appear.

[4] 
https://git.openstack.org/cgit/openstack/fuel-docs/tree/pages/release-notes/v7-0/new_features/plugins.rst?h=stable/7.0

The question of how useful exactly all those plugins are is a bit harder
to answer. DriverLog isn't much help: less than half of Fuel plugins
hosted on OpenStack infrastructure are even registered there, and of
those that are, only 6 have CI jobs with recent successful runs. Does
this mean that 90% of Fuel plugins are broken and unmaintained? Not
necessarily, but it does mean that we have no way to tell.

An even harder question is: once we determine that some plugins are more
equal than others, what should we do about the less useful and the less
actively maintained?

To objectively answer both questions, we need to define support levels
for Fuel plugins and set some reasonable expectations about how plugins
can qualify for each level.

Level 3. Plugin is not actively supported

I believe that having hundreds of Fuel plugins out there on GitHub and
elsewhere is great, and we should encourage people to create more of
those and do whatever they like with them. Even a single-commit "deploy
and forget" plugin is useful as an idea, a source of inspiration, and a
starting point for other people who might want to take it further.

At this level, there should be zero expectations and zero obligations
between Fuel plugin writers and OpenStack community. At the moment, Fuel
plugins developers guide recommends [5] to request a Gerrit repo in the
openstack/ namespace and set up branches, tags, CI, and a code review
process around it, aligned with OpenStack development process. Which is
generally a good idea, except for all the cases where it's too much
overhead and ends up not being followed closely enough to be useful.

[5] https://wiki.openstack.org/wiki/Fuel/Plugins#Repo

Instead of vague blanket recommendations, we should explictly state that
it's fine to do none of that and just stay on GitHub, and that if you
intend to move to the next level and actively maintain your plugin, and
expect support with that from Fuel developers and other OpenStack
projects, these recommendations are not optional and must be fulfilled.

Level 2. Plugin is actively supported by its registered maintainers 

To support a Fuel plugin, we need to answer two fundamental questions:
Can we? Should we?

I think the minimum requirements to say "yes" to both are:

a) All of the plugin's source code is explicitly licensed under an
   OSI-approved license;

b) The plugin source code repository does not contain binary artefacts
   such as RPM packages or ISO images (*);

c) The plugin is registered in DriverLog;

d) Plugin maintainers listed in DriverLog have confirmed the intent to
   support the plugin;

e) Plugin repository on review.openstack.org has a voting CI job that is
   passing with the latest or, at least, previous major release of Fuel.

f) All deviations from the OpenStack development process (alternative
   issue trackers, mailing lists, etc.) are documented in the plugin's
   README file.

*  Aside from purely technical issues we're getting because git is not
   suitable for tracking binary files [6], contaminating the source code
   with opaque binary blobs makes it impossible to ensure that the
   plugin remains compliant with the open source requirement (a).

[6] http://lists.openstack.org/pipermail/openstack-dev/2016-January/083812.html

In addition to above requirements, we need to set up graceful
transitions from level 3 to level 2 and back. Meeting the requirements
should be easy (well, except rewriting commit history to get rid of
binary blobs under .git, I think it's reasonable to require plugin
developers to do this where applicable), and if maintainers step down or
go MIA, we should stash the code in a common repository
(fuel-plugins-contrib) where it can recovered from later.