Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-21 Thread Rob Cresswell 
The aim wasn't to be focused on what's easiest for Horizon at all; it was a 
suggestion to let plugins keep moving during brief periods of instability when 
people make mistakes. We're aware of the plugins need for stability and correct 
deprecation, but we won't catch everything.

That said, there's been significant enough push back against the suggestion 
that I'll rethink it. I'll make sure we discuss the horizon / 
openstack_dashboard split again at the summit too.

Thanks everyone
Rob

On 21 July 2016 at 17:36, David Lyle 
> wrote:
I think basing the plugins against master is the best solution since
the plugin in development will want to work with the matching horizon
release. Finding issues sooner and hopefully one at a time is a lot
easier to address than a bundle of them at the end of a release cycle.
Hopefully, Horizon doesn't break plugins on a regular basis. If we
are, then we are not being focused enough around supporting plugins by
maintaining compatibility and focused too much around what's easiest
for Horizon in the development process. Horizon is no longer at the
top of the stack, we should develop with that awareness. Of course,
breaking changes will happen, but they should be limited and isolated.

David

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-21 Thread David Lyle 
I think basing the plugins against master is the best solution since
the plugin in development will want to work with the matching horizon
release. Finding issues sooner and hopefully one at a time is a lot
easier to address than a bundle of them at the end of a release cycle.
Hopefully, Horizon doesn't break plugins on a regular basis. If we
are, then we are not being focused enough around supporting plugins by
maintaining compatibility and focused too much around what's easiest
for Horizon in the development process. Horizon is no longer at the
top of the stack, we should develop with that awareness. Of course,
breaking changes will happen, but they should be limited and isolated.

David

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-20 Thread Rob Cresswell 
Kirill: The failures are just as visible, since the cores still control merging 
anyway. The only difference is that if it takes a few days to fix something in 
upstream Horizon, you needn't block your own content in the meantime. Later in 
the release cycle (around N-3, for example) cores could just not merge failing 
tests. Regardless, this is just a recommendation, and if you're comfortable 
adjusting to issues as they come through, then thats fine to base off master.

Graham:  The "risk" I refer to is in that in any project architecture rewrite 
you can make mistakes and break functionality. So that risk only arises from a 
rewrite.

"It also means that as a plugin I know that the released version of my plugin 
has been tested in my gate with the exact version of the code that the horizon 
team release." - I don't understand this part. If we release a horizon lib, 
we'd still being doing milestone releases to PyPI. So again, whether you 
consume that as a tarball or a PyPI package makes little difference to the day 
to day testing of your plugin. Its the same code.

Ideally - yes, we'd probably split horizon as a separate library, but thats 
something to discuss at the summit and judge demand, because most discussions 
thus far have concluded that its a lower priority issue than others.

Rob



On 20 July 2016 at 17:36, Hayes, Graham 
> wrote:
On 20/07/2016 15:13, Rob Cresswell wrote:
> Yes, it would mean changing your requirements after a release. So, for
> example you might run two gate tests:
>
> - A voting Horizon-stable/milestone test, (or both)
> - A non-voting Horizon-master test
>
> That gives you a lot of control over making your tests passing (multiple
> patches to make the Horizon-master test pass, or all in one patch set
> alongside the horizon-milestone test bump), rather than random failures
> each week. You'd still be able to track the failures as they come in,
> but they won't break your gate each time.

I don't want control, I want to consume a library in a standard way -
the way we consume libraries from the rest of openstack.

> I don't think where horizon (the library) lives would change how you
> version against it. We don't currently have any plans to separate the
> two; while we realise its a desirable change, weighing the work and risk
> involved in the split architecture vs the end result, we've chosen to
> work on other higher priority items for now (performance, filtering
> improvements, angular work etc.)

Well, if would stop us having a reference to git branch in our
requirements, and allow to use the standard global requirements
process to manage the dependency.

It also means that as a plugin I know that the released version of
my plugin has been tested in my gate with the exact version of the
code that the horizon team release.

We can still do multiple patches to fix any changes in the horizon
library, and in the tip of the chain update the version in requirements.

The risk has just been moved to the plugins, which is not ideal.

It also makes downstream consumption *a lot* easier.

>
>
> On 20 July 2016 at 13:38, Hayes, Graham 
> 
> >> wrote:
>
> On 20/07/2016 10:16, Rob Cresswell wrote:
> > Hey all,
> >
> > So we've had a few issues with plugin stability recently, and its
> > apparent that many plugins are building off Horizon master as a
> > dependency. I would really advise against this. A more manageable
> > development process may be to:
> >
> > - Base stable plugins against a stable release of Horizon
> > - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> > this case, and then bump the version and capture issues within the same
> > patch. This should prevent random breakages, and should allow you to
> > just look at the release notes for that milestone.
>
> So this would mean changing tox.ini or requirements files after each
> horizon release?
>
> This dovetails nicely with the other thread about how we should be doing
> cross project interactions.
>
> What would be best, would be having horizon released as a separate
> library on pypi like the clients and oslo libs.
>
> Then openstack-dashboard, and all the plugins could rely on the same
> base library, without hacks like:
>
>deps =
>  -r{toxinidir}/requirements.txt
>  -r{toxinidir}/test-requirements.txt
>  http://tarballs.openstack.org/horizon/horizon-master.tar.gz
>
> in tox.ini or
>
># Testing Requirements
>
>  http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon
>
> in (test-)requirements.txt
>
> Is that on roadmap?
>
> > On the Horizon side, we've moved our FF a week earlier, so I think that
> > week combined with the usual RC period should be enough time to fix
>

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-20 Thread Hayes, Graham 
On 20/07/2016 15:13, Rob Cresswell wrote:
> Yes, it would mean changing your requirements after a release. So, for
> example you might run two gate tests:
>
> - A voting Horizon-stable/milestone test, (or both)
> - A non-voting Horizon-master test
>
> That gives you a lot of control over making your tests passing (multiple
> patches to make the Horizon-master test pass, or all in one patch set
> alongside the horizon-milestone test bump), rather than random failures
> each week. You'd still be able to track the failures as they come in,
> but they won't break your gate each time.

I don't want control, I want to consume a library in a standard way -
the way we consume libraries from the rest of openstack.

> I don't think where horizon (the library) lives would change how you
> version against it. We don't currently have any plans to separate the
> two; while we realise its a desirable change, weighing the work and risk
> involved in the split architecture vs the end result, we've chosen to
> work on other higher priority items for now (performance, filtering
> improvements, angular work etc.)

Well, if would stop us having a reference to git branch in our
requirements, and allow to use the standard global requirements
process to manage the dependency.

It also means that as a plugin I know that the released version of
my plugin has been tested in my gate with the exact version of the
code that the horizon team release.

We can still do multiple patches to fix any changes in the horizon
library, and in the tip of the chain update the version in requirements.

The risk has just been moved to the plugins, which is not ideal.

It also makes downstream consumption *a lot* easier.

>
>
> On 20 July 2016 at 13:38, Hayes, Graham  > wrote:
>
> On 20/07/2016 10:16, Rob Cresswell wrote:
> > Hey all,
> >
> > So we've had a few issues with plugin stability recently, and its
> > apparent that many plugins are building off Horizon master as a
> > dependency. I would really advise against this. A more manageable
> > development process may be to:
> >
> > - Base stable plugins against a stable release of Horizon
> > - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> > this case, and then bump the version and capture issues within the same
> > patch. This should prevent random breakages, and should allow you to
> > just look at the release notes for that milestone.
>
> So this would mean changing tox.ini or requirements files after each
> horizon release?
>
> This dovetails nicely with the other thread about how we should be doing
> cross project interactions.
>
> What would be best, would be having horizon released as a separate
> library on pypi like the clients and oslo libs.
>
> Then openstack-dashboard, and all the plugins could rely on the same
> base library, without hacks like:
>
>deps =
>  -r{toxinidir}/requirements.txt
>  -r{toxinidir}/test-requirements.txt
>  http://tarballs.openstack.org/horizon/horizon-master.tar.gz
>
> in tox.ini or
>
># Testing Requirements
>
>  http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon
>
> in (test-)requirements.txt
>
> Is that on roadmap?
>
> > On the Horizon side, we've moved our FF a week earlier, so I think that
> > week combined with the usual RC period should be enough time to fix
> > bugs. We'll aim to make sure our release notes are complete with regards
> > to breaking issues for plugins, and deprecate appropriately.
> >
> > Rob
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-20 Thread Kirill Zaitsev 
Initially I don’t think I like the idea of making master-horizon job non-voting 
for murano-dashboard.

Here are some reasons: 
1) We would still need to fix murano-dashboard to work with master 
horizon (since we would need to be released together)
2) The breakage would be less visible
3) I can imagine a situation when a change would pass on master but 
would not pass on a previous stable point release (even worse this release can 
be n3). Say we’re trying to use some function/feature, that has just landed.

Those are my initial ideas about this, have give it a bit more time, to think 
more carefully.

BTW, I can fetch the numbers on how many times murano-dashboard gate was broken 
by changes in horizon, during M and N cycles, if you’re interested. Also for 
murano-dashboard we run our integration selenium tests as a 3d-party CI, so 
technically we’re not blocked by the job not working, in case we need to land 
some security patch. =)

-- 
Kirill Zaitsev
Murano Project Tech Lead
Software Engineer at
Mirantis, Inc

On 20 juillet 2016 at 17:10:46, Rob Cresswell (robert.cressw...@outlook.com) 
wrote:

Yes, it would mean changing your requirements after a release. So, for example 
you might run two gate tests:

- A voting Horizon-stable/milestone test, (or both)
- A non-voting Horizon-master test

That gives you a lot of control over making your tests passing (multiple 
patches to make the Horizon-master test pass, or all in one patch set alongside 
the horizon-milestone test bump), rather than random failures each week. You'd 
still be able to track the failures as they come in, but they won't break your 
gate each time.

I don't think where horizon (the library) lives would change how you version 
against it. We don't currently have any plans to separate the two; while we 
realise its a desirable change, weighing the work and risk involved in the 
split architecture vs the end result, we've chosen to work on other higher 
priority items for now (performance, filtering improvements, angular work etc.)

Rob



On 20 July 2016 at 13:38, Hayes, Graham  wrote:
On 20/07/2016 10:16, Rob Cresswell wrote:
> Hey all,
>
> So we've had a few issues with plugin stability recently, and its
> apparent that many plugins are building off Horizon master as a
> dependency. I would really advise against this. A more manageable
> development process may be to:
>
> - Base stable plugins against a stable release of Horizon
> - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> this case, and then bump the version and capture issues within the same
> patch. This should prevent random breakages, and should allow you to
> just look at the release notes for that milestone.

So this would mean changing tox.ini or requirements files after each
horizon release?

This dovetails nicely with the other thread about how we should be doing
cross project interactions.

What would be best, would be having horizon released as a separate
library on pypi like the clients and oslo libs.

Then openstack-dashboard, and all the plugins could rely on the same
base library, without hacks like:

   deps =
     -r{toxinidir}/requirements.txt
     -r{toxinidir}/test-requirements.txt
     http://tarballs.openstack.org/horizon/horizon-master.tar.gz

in tox.ini or

   # Testing Requirements
   http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon

in (test-)requirements.txt

Is that on roadmap?

> On the Horizon side, we've moved our FF a week earlier, so I think that
> week combined with the usual RC period should be enough time to fix
> bugs. We'll aim to make sure our release notes are complete with regards
> to breaking issues for plugins, and deprecate appropriately.
>
> Rob


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__  
OpenStack Development Mailing List (not for usage questions)  
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe  
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev  


signature.asc
Description: Message signed with OpenPGP using AMPGpg
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-20 Thread Rob Cresswell 
Yes, it would mean changing your requirements after a release. So, for example 
you might run two gate tests:

- A voting Horizon-stable/milestone test, (or both)
- A non-voting Horizon-master test

That gives you a lot of control over making your tests passing (multiple 
patches to make the Horizon-master test pass, or all in one patch set alongside 
the horizon-milestone test bump), rather than random failures each week. You'd 
still be able to track the failures as they come in, but they won't break your 
gate each time.

I don't think where horizon (the library) lives would change how you version 
against it. We don't currently have any plans to separate the two; while we 
realise its a desirable change, weighing the work and risk involved in the 
split architecture vs the end result, we've chosen to work on other higher 
priority items for now (performance, filtering improvements, angular work etc.)

Rob



On 20 July 2016 at 13:38, Hayes, Graham 
> wrote:
On 20/07/2016 10:16, Rob Cresswell wrote:
> Hey all,
>
> So we've had a few issues with plugin stability recently, and its
> apparent that many plugins are building off Horizon master as a
> dependency. I would really advise against this. A more manageable
> development process may be to:
>
> - Base stable plugins against a stable release of Horizon
> - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> this case, and then bump the version and capture issues within the same
> patch. This should prevent random breakages, and should allow you to
> just look at the release notes for that milestone.

So this would mean changing tox.ini or requirements files after each
horizon release?

This dovetails nicely with the other thread about how we should be doing
cross project interactions.

What would be best, would be having horizon released as a separate
library on pypi like the clients and oslo libs.

Then openstack-dashboard, and all the plugins could rely on the same
base library, without hacks like:

   deps =
 -r{toxinidir}/requirements.txt
 -r{toxinidir}/test-requirements.txt
 http://tarballs.openstack.org/horizon/horizon-master.tar.gz

in tox.ini or

   # Testing Requirements
   http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon

in (test-)requirements.txt

Is that on roadmap?

> On the Horizon side, we've moved our FF a week earlier, so I think that
> week combined with the usual RC period should be enough time to fix
> bugs. We'll aim to make sure our release notes are complete with regards
> to breaking issues for plugins, and deprecate appropriately.
>
> Rob


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-20 Thread Hayes, Graham 
On 20/07/2016 10:16, Rob Cresswell wrote:
> Hey all,
>
> So we've had a few issues with plugin stability recently, and its
> apparent that many plugins are building off Horizon master as a
> dependency. I would really advise against this. A more manageable
> development process may be to:
>
> - Base stable plugins against a stable release of Horizon
> - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> this case, and then bump the version and capture issues within the same
> patch. This should prevent random breakages, and should allow you to
> just look at the release notes for that milestone.

So this would mean changing tox.ini or requirements files after each
horizon release?

This dovetails nicely with the other thread about how we should be doing
cross project interactions.

What would be best, would be having horizon released as a separate
library on pypi like the clients and oslo libs.

Then openstack-dashboard, and all the plugins could rely on the same
base library, without hacks like:

   deps =
 -r{toxinidir}/requirements.txt
 -r{toxinidir}/test-requirements.txt
 http://tarballs.openstack.org/horizon/horizon-master.tar.gz

in tox.ini or

   # Testing Requirements
   http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon

in (test-)requirements.txt

Is that on roadmap?

> On the Horizon side, we've moved our FF a week earlier, so I think that
> week combined with the usual RC period should be enough time to fix
> bugs. We'll aim to make sure our release notes are complete with regards
> to breaking issues for plugins, and deprecate appropriately.
>
> Rob


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-20 Thread Vitaly Gridnev 
Hi,

testing against stable release of horizon doesn't have any sense of me.
when you have tests running against master and them are almost always
passing it's much easier to found commit that caused a failure and fix the
problem that it's introduced. anyway, it's great news that FF is moved a
week earlier for horizon, thanks!

On Wed, Jul 20, 2016 at 12:12 PM, Rob Cresswell <
robert.cressw...@outlook.com> wrote:

> Hey all,
>
> So we've had a few issues with plugin stability recently, and its apparent
> that many plugins are building off Horizon master as a dependency. I would
> really advise against this. A more manageable development process may be to:
>
> - Base stable plugins against a stable release of Horizon
> - Base WIP versions/new plugins off the last Horizon milestone, b2 in this
> case, and then bump the version and capture issues within the same patch.
> This should prevent random breakages, and should allow you to just look at
> the release notes for that milestone.
>
> On the Horizon side, we've moved our FF a week earlier, so I think that
> week combined with the usual RC period should be enough time to fix bugs.
> We'll aim to make sure our release notes are complete with regards to
> breaking issues for plugins, and deprecate appropriately.
>
> Rob
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best Regards,
Vitaly Gridnev,
Project Technical Lead of OpenStack DataProcessing Program (Sahara)
Mirantis, Inc
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-20 Thread Rob Cresswell 
Hey all,

So we've had a few issues with plugin stability recently, and its apparent that 
many plugins are building off Horizon master as a dependency. I would really 
advise against this. A more manageable development process may be to:

- Base stable plugins against a stable release of Horizon
- Base WIP versions/new plugins off the last Horizon milestone, b2 in this 
case, and then bump the version and capture issues within the same patch. This 
should prevent random breakages, and should allow you to just look at the 
release notes for that milestone.

On the Horizon side, we've moved our FF a week earlier, so I think that week 
combined with the usual RC period should be enough time to fix bugs. We'll aim 
to make sure our release notes are complete with regards to breaking issues for 
plugins, and deprecate appropriately.

Rob
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev