Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

Thanks, Christopher (and the app-catalog team).

On 21 March 2016 at 08:35, Christopher Aedo  wrote:

> On Sun, Mar 20, 2016 at 1:26 PM, Richard Jones 
> wrote:
> > Unfortunately none of this discussion solves the substantive issue which
> is
> > that we cannot release an xstatic package without breaking the gate.
> >
> > We currently have one solution that's a close to viable as we've been
> able
> > to get: move the version pinning for xstatic packages out of
> > upper-constraints and into Horizon's repository, and hope that the
> > app-catalog server and Horizon stay in step or that they're never
> installed
> > on the same debian/redhat system (or if they *are* then one of them is
> > installed in a virtualenv).
>
> The Horizon plugin for the app catalog[1] should have no problem
> staying in step with Horizon.  Ultimately our intention is to make
> this plugin so desirable it becomes a native part of Horizon so
> enabling in a cloud becomes as simple as a config switch.  We're not
> ready for that yet of course, so please don't let that comment
> distract the thread :)  I am looking forward to discussing this with
> more people in Austin though!
>
> Regardless, we will work with you to make sure our efforts proceed to
> be tightly coupled to Horizon.  This approach makes the most sense to
> me.
>
> [1]: http://git.openstack.org/cgit/openstack/app-catalog-ui/
>
> -Christopher
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Christopher Aedo]

On Sun, Mar 20, 2016 at 1:26 PM, Richard Jones  wrote:
> Unfortunately none of this discussion solves the substantive issue which is
> that we cannot release an xstatic package without breaking the gate.
>
> We currently have one solution that's a close to viable as we've been able
> to get: move the version pinning for xstatic packages out of
> upper-constraints and into Horizon's repository, and hope that the
> app-catalog server and Horizon stay in step or that they're never installed
> on the same debian/redhat system (or if they *are* then one of them is
> installed in a virtualenv).

The Horizon plugin for the app catalog[1] should have no problem
staying in step with Horizon.  Ultimately our intention is to make
this plugin so desirable it becomes a native part of Horizon so
enabling in a cloud becomes as simple as a config switch.  We're not
ready for that yet of course, so please don't let that comment
distract the thread :)  I am looking forward to discussing this with
more people in Austin though!

Regardless, we will work with you to make sure our efforts proceed to
be tightly coupled to Horizon.  This approach makes the most sense to
me.

[1]: http://git.openstack.org/cgit/openstack/app-catalog-ui/

-Christopher

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

Unfortunately none of this discussion solves the substantive issue which is
that we cannot release an xstatic package without breaking the gate.

We currently have one solution that's a close to viable as we've been able
to get: move the version pinning for xstatic packages out of
upper-constraints and into Horizon's repository, and hope that the
app-catalog server and Horizon stay in step or that they're never installed
on the same debian/redhat system (or if they *are* then one of them is
installed in a virtualenv).


 Richard


On 17 March 2016 at 19:00, Thomas Goirand  wrote:

> On 03/17/2016 07:23 AM, Richard Jones wrote:
> > There's a basic difference here though. Your traditional "installed
> > components" are pieces of software and data used *by programs on that
> > system.*
> >
> > The components we're talking about here are, as far as the system is
> > concerned, opaque data to be transmitted over HTTP(S) to a web browser
> > client which then makes use of that data in some manner.
> >
> > There are no cross-program compatibility issues stemming from having
> > multiple different versioned copies of such client-side files on a
> > system
>
> The same way, we could have multiple version of fonts, tzdata, SSL root
> certificates and so on. There wouldn't be any compatibility issues.
> Though it's still not the right thing to do at a distribution level.
>
> Have you noticed also that in the Windows world, each program carries
> its .dll, which are supposed to be shared object, but in fact, they
> aren't shared? Yes, it is easier to do so.
>
> > - this is why the web development world has standardised on
> > tooling that *makes it easy to do so*. Different client-side web
> > applications *should* be able to use different versions of components.
>
> The same was as for shared .so libraries, that's not the correct way to
> do things. Even though the JavaScript objects aren't executed by the
> system (well, if we forget that nodejs exists), there's still potential
> bugs and security problems with them, and they may require maintenance.
>
> > xstatic shoe-horns that freedom of client-side application component
> > usage into a one-size-must-fit-all world that fundamentally only exists
> > because programs on a system can get confused when multiple versions are
> > installed on that system[1].
>
> I wouldn't say it this way. To me, they are just tools which makes it
> easy for us to stop the duplication madness of the same files.
>
> Have a look:
> # apt-file search jquery.js | grep -v doc | wc -l
> 128
>
> This is 127 bugs which should be fixed currently with the embedded
> jquery. I hardly see how one could argue this is a good thing. I hope we
> can be better citizens than this.
>
> Cheers,
>
> Thomas Goirand (zigo)
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 13 March 2016 at 07:06, Matthias Runge  wrote:

> On 10/03/16 08:46, Richard Jones wrote:
> > On 10 March 2016 at 18:23, Matthias Runge  > > wrote:
> >
> > 4.alt.2:
> > move to proper packages for static file management. I mean, they
> need to
> > be built anyways.
> >
> >
> > Please define what you mean by "proper packages" here. I *think* you
> > might mean system packages (aka Debian or Red Hat) which is not feasible
> > given other environments that Horizon runs under. Please correct me if
> > I'm wrong!
>
> Exactly. I mean system packages. If there are issues with system
> packages, then let's address the issue rather than re-inventing the wheel.
>

OK, the sticking point is that installation through system packages alone
forces us to make all software on a system work with a single version of
any given library. This has spawned the global-requirements and now
upper-constraints systems in OpenStack, and ultimately leads to the
problematic race condition that resulted in me starting this email thread.
That is, we can update *either* the version of a library being used *or*
the software that is compatible with that version *but not the two at the
same time, atomically*.



> Weren't we just talking about providing dependencies for the gate?


Well, partially, because the reason the problem surfaces is because of the
Continuous Deployment guarantee that OpenStack makes, which is enforced by
the gate, so sure, let's say it's the gate's fault 



> I mean, for production, it's quite the same situation we are at the
> moment. Nobody requires you to install Horizon and dependencies
> especially via rpm, deb or pip: Take what you want.
>

I'm not sure it's this simple, is it? Folks want to be able to install
OpenStack from a single source, and that seems reasonable. They want to be
able to do that "offline" too, so that rules out bower as a source of
packages.



> > It has been mentioned, xstatic packages can block the gate. We
> currently
> > control xstatic package releases, thus we can roll back, if something
> > goes wrong.
> >
> > If we're pulling directly with npm/bower, someone from the outside
> can
> > break us. We already have the situation with pypi packages.
> > With proper packages, we could even use the gate to release those
> > packages and thus verify, we're not breaking anyone.
> >
> >
> > We're going to have potential breakage (gate breakage, in the integrated
> > tests) any time we release a package (regardless of release mechanism)
> > and have to update two separate repositories resulting in out-of-sync
> > version specification and expectation (ie. upper-constraints
> > specification and Horizon's code expectation) as described in my OP. The
> > only solution that we're aware of is to synchronise updating those two
> > things, through one of the mechanisms proposed so far (or possibly
> > through a mechanism not yet proposed.)
> >
>
> Yes, and my proposal to address this is to gate updating/releasing
> dependencies the same way we're currently gating each change in horizon.
>

This is not going to solve the race condition I mention; it's actually
during our work implementing gating these releases that we found we had to
solve this problem.



> > 1. Horizon maintains its own constrained version list for the xstatic
> > packages,
> > 2. Plugins to Horizon must depend on Horizon to supply xstatic packages
> > except where they use additional packages that Horizon does not use, and
> > 3. We avoid installing app-catalog (the system, not the plugin) in the
> > integrated tests (I don't believe doing this is even on the ...
> > "horizon" so to speak) *or* in a Debian / Red Hat (i.e. system-packaged)
> > system that also has Horizon installed. Or we try to convince
> > app-catalog to stay lock-step with Horizon's xstatic versions. I
> > understand the risk of a collision between app-catalog and Horizon in
> > the same system-packaged environment is very low.
>
> I don't really see a chance for app-catalog to require Horizon as a
> dependency and different versions of xstatic packages. This would be an
> immediate show-stopper for app-catalog either on Debian or on RPM based
> systems.
>

I think I used the wrong word above. Where I said "system" I probably
should have said "server". app-catalog the stand-alone server should not
depend on Horizon, just app-catalog the plugin to Horizon should (like all
Horizon plugins should).



> Let me repeat myself: you're free to install dependencies as you like,
> npm, bower, whatever. I was simply speaking about the gate and about
> gating dependencies to be sure, we're not broken by someone from outside.


Again, I don't believe we have the freedom to actually install dependencies
as we like, as I said above.


  Richard
__
OpenStack Development Mailing List (not for 

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Thomas Goirand]

On 03/17/2016 07:12 AM, Richard Jones wrote:
> On 13 March 2016 at 07:11, Matthias Runge  > wrote:
> 
> On 10/03/16 11:48, Beth Elwell wrote:
> > If we will anyway have potential breakage I don’t understand why the
> > better solution here would not be to just use the bower and npm tools
> > which are standardised for JavaScript and would move Horizon more
> > towards using widely recognised tooling from within not just Openstack
> > but the wider development community. Back versions always need to be
> > supported for a time, however I would add that long term this could end
> > up saving time and create a stable longer term solution.
> >
> 
> I have a few issues with those "package managers":
> - downloads are not verified, there is a chance of getting a "bad"
> download.
> - they are pointing to the outside world, like to github etc. While they
> appear to work "most of the time", that might not good enough for
> the gate
> - how often have we been blocked by releases of software not managed by
> OpenStack? Seriously, that happens quite a few times over a release
> cycle, not to mention breakages by releases of our own tools turning out
> to block one or the other sub-project
> 
> 
> To be fair to those package managers,  the issues OpenStack has had with
> releases of libraries breaking things is a result of us either:
> 
> a) not pinning releases (upper-constraints now fixes that for *things
> that use it*, which isn't everything, sadly) or
> b) the system that tests upper-constraints changes not having broad
> enough testing across OpenStack for us to notice when a new library
> release breaks things. I would like to increase the inclusion of
> Horizon's test suite in the constraints testing for this reason. At
> least, it's on my TODO :-)
> 
> Horizon, for example, currently does *not* use the upper-constraints
> pinning in its test suite or installation, so we're vulnerable to, say,
> a python-*client release that's not compatible. I have a patch in the
> works to address this, but it kinda depends on us moving over from
> run_tests.sh to tox, which is definitely something to wait until N for.

Thanks for this work. I'm looking forward to it. Do you also have in the
pipe to stop using nose / python -m coverage, and switch to testr?

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 14 March 2016 at 11:08, Thomas Goirand  wrote:
>
> From a distribution package maintainer perspective, the most annoying
> part is that there's no easy way to get the web app use the JS libs from
> the OS, and there's no system wide registry of installed components.
>

There's a basic difference here though. Your traditional "installed
components" are pieces of software and data used *by programs on that
system.*

The components we're talking about here are, as far as the system is
concerned, opaque data to be transmitted over HTTP(S) to a web browser
client which then makes use of that data in some manner.

There are no cross-program compatibility issues stemming from having
multiple different versioned copies of such client-side files on a system -
this is why the web development world has standardised on tooling that
*makes it easy to do so*. Different client-side web applications *should*
be able to use different versions of components.

xstatic shoe-horns that freedom of client-side application component usage
into a one-size-must-fit-all world that fundamentally only exists because
programs on a system can get confused when multiple versions are installed
on that system[1].


Richard

[1] I note that OS X packaging solved that problem too, but let's not go
there 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Matthias Runge]

On 17/03/16 08:59, Thomas Goirand wrote:

>>
>> Horizon, for example, currently does *not* use the upper-constraints
>> pinning in its test suite or installation, so we're vulnerable to, say,
>> a python-*client release that's not compatible. I have a patch in the
>> works to address this, but it kinda depends on us moving over from
>> run_tests.sh to tox, which is definitely something to wait until N for.
> 
> Thanks for this work. I'm looking forward to it. Do you also have in the
> pipe to stop using nose / python -m coverage, and switch to testr?

There is basic testr support[1], and Itxaka has a patch up[2], to
replace run_tests.sh completely with tox.



[1]
https://github.com/openstack/horizon/commit/ab9e5d94af2b15eef39f8f971a7a0c9778883a4b
[2] https://review.openstack.org/#/c/259013/
-- 
Matthias Runge 

Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham,
Michael O'Neill

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Thomas Goirand]

On 03/17/2016 07:23 AM, Richard Jones wrote:
> There's a basic difference here though. Your traditional "installed
> components" are pieces of software and data used *by programs on that
> system.*
> 
> The components we're talking about here are, as far as the system is
> concerned, opaque data to be transmitted over HTTP(S) to a web browser
> client which then makes use of that data in some manner.
> 
> There are no cross-program compatibility issues stemming from having
> multiple different versioned copies of such client-side files on a
> system

The same way, we could have multiple version of fonts, tzdata, SSL root
certificates and so on. There wouldn't be any compatibility issues.
Though it's still not the right thing to do at a distribution level.

Have you noticed also that in the Windows world, each program carries
its .dll, which are supposed to be shared object, but in fact, they
aren't shared? Yes, it is easier to do so.

> - this is why the web development world has standardised on
> tooling that *makes it easy to do so*. Different client-side web
> applications *should* be able to use different versions of components.

The same was as for shared .so libraries, that's not the correct way to
do things. Even though the JavaScript objects aren't executed by the
system (well, if we forget that nodejs exists), there's still potential
bugs and security problems with them, and they may require maintenance.

> xstatic shoe-horns that freedom of client-side application component
> usage into a one-size-must-fit-all world that fundamentally only exists
> because programs on a system can get confused when multiple versions are
> installed on that system[1].

I wouldn't say it this way. To me, they are just tools which makes it
easy for us to stop the duplication madness of the same files.

Have a look:
# apt-file search jquery.js | grep -v doc | wc -l
128

This is 127 bugs which should be fixed currently with the embedded
jquery. I hardly see how one could argue this is a good thing. I hope we
can be better citizens than this.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 13 March 2016 at 07:11, Matthias Runge  wrote:

> On 10/03/16 11:48, Beth Elwell wrote:
> > If we will anyway have potential breakage I don’t understand why the
> > better solution here would not be to just use the bower and npm tools
> > which are standardised for JavaScript and would move Horizon more
> > towards using widely recognised tooling from within not just Openstack
> > but the wider development community. Back versions always need to be
> > supported for a time, however I would add that long term this could end
> > up saving time and create a stable longer term solution.
> >
>
> I have a few issues with those "package managers":
> - downloads are not verified, there is a chance of getting a "bad"
> download.
> - they are pointing to the outside world, like to github etc. While they
> appear to work "most of the time", that might not good enough for the gate
> - how often have we been blocked by releases of software not managed by
> OpenStack? Seriously, that happens quite a few times over a release
> cycle, not to mention breakages by releases of our own tools turning out
> to block one or the other sub-project


To be fair to those package managers,  the issues OpenStack has had with
releases of libraries breaking things is a result of us either:

a) not pinning releases (upper-constraints now fixes that for *things that
use it*, which isn't everything, sadly) or
b) the system that tests upper-constraints changes not having broad enough
testing across OpenStack for us to notice when a new library release breaks
things. I would like to increase the inclusion of Horizon's test suite in
the constraints testing for this reason. At least, it's on my TODO :-)

Horizon, for example, currently does *not* use the upper-constraints
pinning in its test suite or installation, so we're vulnerable to, say, a
python-*client release that's not compatible. I have a patch in the works
to address this, but it kinda depends on us moving over from run_tests.sh
to tox, which is definitely something to wait until N for.


 Richard

ps. as for "unverified downloads" ... they're verified precisely as much as
pypi packages are, and we install a whole buncha those :-)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Thomas Goirand]

On 03/12/2016 09:11 PM, Matthias Runge wrote:
> - how often have we been blocked by releases of software not managed by
> OpenStack? Seriously, that happens quite a few times over a release
> cycle, not to mention breakages by releases of our own tools turning out
> to block one or the other sub-project.

Let's remember the bad releases of setuptools :)

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Thomas Goirand]

On 03/10/2016 12:14 PM, Richard Jones wrote:
> On 10 March 2016 at 21:48, Beth Elwell  > wrote:
> 
>> On 10 Mar 2016, at 07:46, Richard Jones > > wrote:
>>
>> It has been mentioned, xstatic packages can block the gate. We
>> currently
>> control xstatic package releases, thus we can roll back, if
>> something
>> goes wrong.
>>
>> If we're pulling directly with npm/bower, someone from the
>> outside can
>> break us. We already have the situation with pypi packages.
>> With proper packages, we could even use the gate to release those
>> packages and thus verify, we're not breaking anyone.
>>
>>
>> We're going to have potential breakage (gate breakage, in the
>> integrated tests) any time we release a package (regardless of
>> release mechanism) and have to update two separate repositories
>> resulting in out-of-sync version specification
>> and expectation (ie. upper-constraints specification and Horizon's
>> code expectation) as described in my OP. The only solution that
>> we're aware of is to synchronise updating those two things,
>> through one of the mechanisms proposed so far (or possibly through
>> a mechanism not yet proposed.)
> If we will anyway have potential breakage I don’t understand why the
> better solution here would not be to just use the bower and npm
> tools which are standardised for JavaScript and would move Horizon
> more towards using widely recognised tooling from within not just
> Openstack but the wider development community. Back versions always
> need to be supported for a time, however I would add that long term
> this could end up saving time and create a stable longer term solution.
> 
> 
> I (and others) have argued several times for this, for a number of
> reasons, and it remains my preferred solution, but it has been shot down
> many times :-(

And I will attempt to do it once more unless someone brings the right
argumentation to the table.

> There are three major hurdles that I recall (sorry if I forgot any, it's
> getting late here):
> 
> 1. system packaging; installers using Debian or Red Hat (completely
> offline) installation will not be able to install Horizon. We don't have
> a solution for this though various caching mechanisms have been proposed.
> 2. security; the bower installation mechanism is seen as being very
> insecure - not that I would call the current xstatic mechanism secure.
> It's all down to degrees, I suppose.
> 3. installation in the gate; I believe that currently installation from
> bower would not be possible in the gate. This is pretty closely related
> to #1.
> 
> I think I recall licensing came up at one point too, but I might be
> mis-remembering.
> 
>   Richard

>From a distribution package maintainer perspective, the most annoying
part is that there's no easy way to get the web app use the JS libs from
the OS, and there's no system wide registry of installed components.
XStatic does that. Get $your-favoring-js-package-manager to do it, just
like php pear, perl CPAN, or Python pip does, then we will adopt it.
This, of course, requires work on upstream tooling.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Thomas Goirand]

On 03/10/2016 11:48 AM, Beth Elwell wrote:
> 
>> On 10 Mar 2016, at 07:46, Richard Jones > > wrote:
>>  
>>
>> It has been mentioned, xstatic packages can block the gate. We
>> currently
>> control xstatic package releases, thus we can roll back, if something
>> goes wrong.
>>
>> If we're pulling directly with npm/bower, someone from the outside can
>> break us. We already have the situation with pypi packages.
>> With proper packages, we could even use the gate to release those
>> packages and thus verify, we're not breaking anyone.
>>
>>
>> We're going to have potential breakage (gate breakage, in the
>> integrated tests) any time we release a package (regardless of release
>> mechanism) and have to update two separate repositories resulting in
>> out-of-sync version specification and expectation (ie.
>> upper-constraints specification and Horizon's code expectation) as
>> described in my OP. The only solution that we're aware of is to
>> synchronise updating those two things, through one of the mechanisms
>> proposed so far (or possibly through a mechanism not yet proposed.)
> 
> If we will anyway have potential breakage I don’t understand why the
> better solution here would not be to just use the bower and npm tools

We control the breakages, and we're trying to find a solution (if you
didn't notice, many where proposed in this thread).

> which are standardised for JavaScript

They are a *very bad* standard which doesn't work at all with downstream
distributions. Unless someone adds a system wide registry for javascript
(the say way that if you apt-get a Python package, pip wont download
it), then we can't use it.

> and would move Horizon more
> towards using widely recognised tooling

Recognized by who? Certainly not downstream distributions. This is a
nightmare.

> from within not just Openstack
> but the wider development community.

What community are you talking about? That JavaScript community which
breaks APIs every 2 weeks? Please re-read this:

https://github.com/openstack/horizon/blob/master/doc/source/topics/packaging.rst

and especially the part:
"Why we use XStatic"

We have every few months some Horizon contributors advocating for
pushing toward this direction, however, none have yet shown a way to
solve what Xstatic does. Until this is done, please refrain from writing:

"hey, everyone uses X, so let's do the same"

without valid technical argumentation.

Yes, using npm / bower / gulp / you-name-it, may make your Horizon
contributor life easier. But that's *not* the only thing to consider.

> Back versions always need to be
> supported for a time, however I would add that long term this could end
> up saving time and create a stable longer term solution.

Long term, I would like to see more contributions to the upstream
JavaScript tooling to make it behave reasonably. Until this is done, we
can't use it.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Thomas Goirand]

On 03/09/2016 04:53 PM, Monty Taylor wrote:
> On 03/07/2016 10:52 PM, Richard Jones wrote:
>> We've solved *most* of the issues around releasing new xstatic packages,
>> documented here[1] (and related documentation).
>>
>> We have one final issue that's blocking us, which is that during the
>> xstatic release there will be a point at which Horizon may be broken
>> from an integrated point of view - some of the interfaces may not work
>> and fail tests. The process goes something like this:
>>
>> ​Note: this assumes that depends-on can reliably bring in patches from
>> all over the place into a gate environment, which is technically
>> possible, but not necessarily correct today.
>>
>> The problem is that because we can't atomically update both
>> upper-constraints *and* Horizon at the same time (or upper-constraints
>> *and* xstatic-angular, or Horizon *and* xstatic-angular) we run into a
>> situation where Horizon will be running against the wrong version of
>> xstatic-angular.
>>
>> So we break one of the basic assumptions of the OpenStack world: that
>> every single commit in every repository for the integrated environment
>> will pass tests.
>>
>> In the Python world, we code around this by making Horizon compatible
>> with both the X and X1 versions of xstatic-angular (if it was a Python
>> library). In Javascript land this is much more difficult - Javascript
>> libraries tend to break compatibility in far more interesting ways.
> 
> Yah. Honestly, this is one of the main places where I think we get into
> trouble in linux-distro land in trying to apply the tools/techniques
> developed for one set of problems with another.
> 
>> Maintaining compatibility across CSS and font releases is also a
>> difficult problem, though changes here are unlikely to break Horizon
>> enough that gate tests would fail. So, solution 1 to the problem is:
>>
>> SOLUTION 1: always maintain Horizon compatibility across xstatic library
>> releases.
>>
>> This is potentially very difficult to guarantee. So, a second solution
>> has been proposed:
>>
>> SOLUTION 2: move the upper-constraints information for *the xstatic
>> libraries only* from the global upper-constraints file into Horizon's
>> repository.
>>
>> This allows us to atomically update both Horizon and the xstatic library
>> version, removing the potential to break because of the version
>> mismatch. Unfortunately it means that we have version compatibility
>> issues with anything else that wants to install alongside Horizon that
>> also uses xstatic packages. For example, Horizon plugins. We could move
>> Horizon plugins into the Horizon repository to solve this. /ducks
> 
> I actually like this one a lot. I know that there is a plugin problem
> ... but ... plugin installers could also consume the horizon xstatic
> constraints when installing xstatic packages?
> 
> xstatic packages are specifically workarounds for doing javascript in a
> python-centric world. Putting then in upper-constraints and actually
> treating them like actual python packages rather than what they are
> which is a clever utilization of the python ecosystem to deliver
> javascript libraries is vexing at best.
> 
> 
>> A variation on this solution is:
>>
>> SOLUTION 3: allow Horizon to locally override upper-constraints for the
>> time needed to merge a patch in devstack.
>>
>> This solution allows Horizon to atomically update itself and the xstatic
>> library, but it also means installing Horizon in a CI/CD environment
>> becomes more difficult due to the need to always pull down the
>> upper-constraints file and edit it. We could code this in to tox, but
>> that doesn't help eg. devstack which needs to also do this thing.
> 
> Or people who are deploying horizon CD from master from source and
> applying upper-constraints to get the tested version. Those people would
> have to duplicate the tox logic.
> 
>> SOLUTION 4: vendor the javascript
>>
>> Heh.
> 
> Heh indeed.
> 
> SOLUTION 4.alt: use npm/bower instead of xstatic to pull the javascript.

Veto. See "Why we use XStatic" at:
https://github.com/openstack/horizon/blob/master/doc/source/topics/packaging.rst

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Thomas Goirand]

On 03/08/2016 05:52 AM, Richard Jones wrote:
> In the Python world, we code around this by making Horizon compatible
> with both the X and X1 versions of xstatic-angular (if it was a Python
> library). In Javascript land this is much more difficult - Javascript
> libraries tend to break compatibility in far more interesting ways.
> Maintaining compatibility across CSS and font releases is also a
> difficult problem, though changes here are unlikely to break Horizon
> enough that gate tests would fail. So, solution 1 to the problem is:
> 
> SOLUTION 1: always maintain Horizon compatibility across xstatic library
> releases.
> 
> This is potentially very difficult to guarantee. So, a second solution
> has been proposed:
> 
> SOLUTION 2: move the upper-constraints information for *the xstatic
> libraries only* from the global upper-constraints file into Horizon's
> repository.
> 
> This allows us to atomically update both Horizon and the xstatic library
> version, removing the potential to break because of the version
> mismatch. Unfortunately it means that we have version compatibility
> issues with anything else that wants to install alongside Horizon that
> also uses xstatic packages. For example, Horizon plugins. We could move
> Horizon plugins into the Horizon repository to solve this. /ducks
> 
> A variation on this solution is:
> 
> SOLUTION 3: allow Horizon to locally override upper-constraints for the
> time needed to merge a patch in devstack.
> 
> This solution allows Horizon to atomically update itself and the xstatic
> library, but it also means installing Horizon in a CI/CD environment
> becomes more difficult due to the need to always pull down the
> upper-constraints file and edit it. We could code this in to tox, but
> that doesn't help eg. devstack which needs to also do this thing.
> 
> SOLUTION 4: vendor the javascript
> 
> Heh.
> 
> SOLUTION 5: have dependencies on xstatic move from global requirements
> to Horizon
> 
> This is similar to 2 & 3 with some of the same drawbacks (multiple users
> of xstatic) but also we'd need a change to global-requirements handling
> to ignore some entries in Horizon's requirements.
> 
> Your thoughts on any and all of the above are requested.

SOLUTION 6: educate upstream and make them stop breaking the world.

SOLUTION 7: Stop having dependencies on libraries that constantly break
the world.

SOLUTION 8: Stop using JavaScript.

SOLUTION 9: Rewrite Horizon as QT client! :)

Cheers,

Thomas Goirand (zigo)

P.S: While all of this is just for fun, I seriously believe that
upstream JavaScript people should be educated to not break things every
2 weeks... though I don't think it will happen anytime soon.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Matthias Runge]

On 10/03/16 11:48, Beth Elwell wrote:

> 
> If we will anyway have potential breakage I don’t understand why the
> better solution here would not be to just use the bower and npm tools
> which are standardised for JavaScript and would move Horizon more
> towards using widely recognised tooling from within not just Openstack
> but the wider development community. Back versions always need to be
> supported for a time, however I would add that long term this could end
> up saving time and create a stable longer term solution.
> 

I have a few issues with those "package managers":
- downloads are not verified, there is a chance of getting a "bad" download.
- they are pointing to the outside world, like to github etc. While they
appear to work "most of the time", that might not good enough for the gate
- how often have we been blocked by releases of software not managed by
OpenStack? Seriously, that happens quite a few times over a release
cycle, not to mention breakages by releases of our own tools turning out
to block one or the other sub-project.

Matthias
-- 
Matthias Runge 

Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham,
Michael O'Neill

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Matthias Runge]

On 10/03/16 08:46, Richard Jones wrote:
> On 10 March 2016 at 18:23, Matthias Runge  > wrote:
> 
> 4.alt.2:
> move to proper packages for static file management. I mean, they need to
> be built anyways.
> 
> 
> Please define what you mean by "proper packages" here. I *think* you
> might mean system packages (aka Debian or Red Hat) which is not feasible
> given other environments that Horizon runs under. Please correct me if
> I'm wrong!

Exactly. I mean system packages. If there are issues with system
packages, then let's address the issue rather than re-inventing the wheel.

Weren't we just talking about providing dependencies for the gate? I
mean, for production, it's quite the same situation we are at the
moment. Nobody requires you to install Horizon and dependencies
especially via rpm, deb or pip: Take what you want.

> It has been mentioned, xstatic packages can block the gate. We currently
> control xstatic package releases, thus we can roll back, if something
> goes wrong.
> 
> If we're pulling directly with npm/bower, someone from the outside can
> break us. We already have the situation with pypi packages.
> With proper packages, we could even use the gate to release those
> packages and thus verify, we're not breaking anyone.
> 
> 
> We're going to have potential breakage (gate breakage, in the integrated
> tests) any time we release a package (regardless of release mechanism)
> and have to update two separate repositories resulting in out-of-sync
> version specification and expectation (ie. upper-constraints
> specification and Horizon's code expectation) as described in my OP. The
> only solution that we're aware of is to synchronise updating those two
> things, through one of the mechanisms proposed so far (or possibly
> through a mechanism not yet proposed.)
> 

Yes, and my proposal to address this is to gate updating/releasing
dependencies the same way we're currently gating each change in horizon.

> 
> 1. Horizon maintains its own constrained version list for the xstatic
> packages,
> 2. Plugins to Horizon must depend on Horizon to supply xstatic packages
> except where they use additional packages that Horizon does not use, and
> 3. We avoid installing app-catalog (the system, not the plugin) in the
> integrated tests (I don't believe doing this is even on the ...
> "horizon" so to speak) *or* in a Debian / Red Hat (i.e. system-packaged)
> system that also has Horizon installed. Or we try to convince
> app-catalog to stay lock-step with Horizon's xstatic versions. I
> understand the risk of a collision between app-catalog and Horizon in
> the same system-packaged environment is very low.

I don't really see a chance for app-catalog to require Horizon as a
dependency and different versions of xstatic packages. This would be an
immediate show-stopper for app-catalog either on Debian or on RPM based
systems.

Let me repeat myself: you're free to install dependencies as you like,
npm, bower, whatever. I was simply speaking about the gate and about
gating dependencies to be sure, we're not broken by someone from outside.

Matthias
-- 
Matthias Runge 

Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham,
Michael O'Neill

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 10 March 2016 at 21:48, Beth Elwell  wrote:
>
> On 10 Mar 2016, at 07:46, Richard Jones  wrote:
>
>> It has been mentioned, xstatic packages can block the gate. We currently
>> control xstatic package releases, thus we can roll back, if something
>> goes wrong.
>>
>> If we're pulling directly with npm/bower, someone from the outside can
>> break us. We already have the situation with pypi packages.
>> With proper packages, we could even use the gate to release those
>> packages and thus verify, we're not breaking anyone.
>>
>
> We're going to have potential breakage (gate breakage, in the integrated
> tests) any time we release a package (regardless of release mechanism) and
> have to update two separate repositories resulting in out-of-sync version
> specification and expectation (ie. upper-constraints specification and
> Horizon's code expectation) as described in my OP. The only solution that
> we're aware of is to synchronise updating those two things, through one of
> the mechanisms proposed so far (or possibly through a mechanism not yet
> proposed.)
>
> If we will anyway have potential breakage I don’t understand why the
> better solution here would not be to just use the bower and npm tools which
> are standardised for JavaScript and would move Horizon more towards using
> widely recognised tooling from within not just Openstack but the wider
> development community. Back versions always need to be supported for a
> time, however I would add that long term this could end up saving time and
> create a stable longer term solution.
>

I (and others) have argued several times for this, for a number of reasons,
and it remains my preferred solution, but it has been shot down many times
:-(

There are three major hurdles that I recall (sorry if I forgot any, it's
getting late here):

1. system packaging; installers using Debian or Red Hat (completely
offline) installation will not be able to install Horizon. We don't have a
solution for this though various caching mechanisms have been proposed.
2. security; the bower installation mechanism is seen as being very
insecure - not that I would call the current xstatic mechanism secure. It's
all down to degrees, I suppose.
3. installation in the gate; I believe that currently installation from
bower would not be possible in the gate. This is pretty closely related to
#1.

I think I recall licensing came up at one point too, but I might be
mis-remembering.


  Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Itxaka Serrano Garcia]

On 03/10/2016 11:48 AM, Beth Elwell wrote:



On 10 Mar 2016, at 07:46, Richard Jones > wrote:

It has been mentioned, xstatic packages can block the gate. We
currently
control xstatic package releases, thus we can roll back, if something
goes wrong.

If we're pulling directly with npm/bower, someone from the outside can
break us. We already have the situation with pypi packages.
With proper packages, we could even use the gate to release those
packages and thus verify, we're not breaking anyone.


We're going to have potential breakage (gate breakage, in the
integrated tests) any time we release a package (regardless of release
mechanism) and have to update two separate repositories resulting in
out-of-sync version specification and expectation (ie.
upper-constraints specification and Horizon's code expectation) as
described in my OP. The only solution that we're aware of is to
synchronise updating those two things, through one of the mechanisms
proposed so far (or possibly through a mechanism not yet proposed.)


If we will anyway have potential breakage I don’t understand why the
better solution here would not be to just use the bower and npm tools
which are standardised for JavaScript and would move Horizon more
towards using widely recognised tooling from within not just Openstack
but the wider development community. Back versions always need to be
supported for a time, however I would add that long term this could end
up saving time and create a stable longer term solution.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org
?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




npm/bower seems like the right tool for this instead of trying to 
convert everything to the python ecosystem.



And I dont understand the issues with plugins. They depend on a horizon 
version so they need to work with the js libraries that are provided by 
that version, same as with any python packages that horizon brings, they 
have to work with that, should not be a difference in there no?


Itxaka



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Beth Elwell]

> On 10 Mar 2016, at 07:46, Richard Jones  wrote:
>  
> It has been mentioned, xstatic packages can block the gate. We currently
> control xstatic package releases, thus we can roll back, if something
> goes wrong.
> 
> If we're pulling directly with npm/bower, someone from the outside can
> break us. We already have the situation with pypi packages.
> With proper packages, we could even use the gate to release those
> packages and thus verify, we're not breaking anyone.
> 
> We're going to have potential breakage (gate breakage, in the integrated 
> tests) any time we release a package (regardless of release mechanism) and 
> have to update two separate repositories resulting in out-of-sync version 
> specification and expectation (ie. upper-constraints specification and 
> Horizon's code expectation) as described in my OP. The only solution that 
> we're aware of is to synchronise updating those two things, through one of 
> the mechanisms proposed so far (or possibly through a mechanism not yet 
> proposed.)

If we will anyway have potential breakage I don’t understand why the better 
solution here would not be to just use the bower and npm tools which are 
standardised for JavaScript and would move Horizon more towards using widely 
recognised tooling from within not just Openstack but the wider development 
community. Back versions always need to be supported for a time, however I 
would add that long term this could end up saving time and create a stable 
longer term solution.

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 10 March 2016 at 18:23, Matthias Runge  wrote:
>
> 4.alt.2:
> move to proper packages for static file management. I mean, they need to
> be built anyways.
>

Please define what you mean by "proper packages" here. I *think* you might
mean system packages (aka Debian or Red Hat) which is not feasible given
other environments that Horizon runs under. Please correct me if I'm wrong!



> It has been mentioned, xstatic packages can block the gate. We currently
> control xstatic package releases, thus we can roll back, if something
> goes wrong.
>
> If we're pulling directly with npm/bower, someone from the outside can
> break us. We already have the situation with pypi packages.
> With proper packages, we could even use the gate to release those
> packages and thus verify, we're not breaking anyone.
>

We're going to have potential breakage (gate breakage, in the integrated
tests) any time we release a package (regardless of release mechanism) and
have to update two separate repositories resulting in out-of-sync version
specification and expectation (ie. upper-constraints specification and
Horizon's code expectation) as described in my OP. The only solution that
we're aware of is to synchronise updating those two things, through one of
the mechanisms proposed so far (or possibly through a mechanism not yet
proposed.)


I think that David's proposal is the only really feasible approach at this
time:

1. Horizon maintains its own constrained version list for the xstatic
packages,
2. Plugins to Horizon must depend on Horizon to supply xstatic packages
except where they use additional packages that Horizon does not use, and
3. We avoid installing app-catalog (the system, not the plugin) in the
integrated tests (I don't believe doing this is even on the ... "horizon"
so to speak) *or* in a Debian / Red Hat (i.e. system-packaged) system that
also has Horizon installed. Or we try to convince app-catalog to stay
lock-step with Horizon's xstatic versions. I understand the risk of a
collision between app-catalog and Horizon in the same system-packaged
environment is very low.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Matthias Runge]

On 09/03/16 19:52, Michael Krotscheck wrote:
> On Wed, Mar 9, 2016 at 7:58 AM Monty Taylor  > wrote:
> 
> 
> > SOLUTION 4: vendor the javascript
> >
> > Heh.
> 
> Heh indeed.
> 
> SOLUTION 4.alt: use npm/bower instead of xstatic to pull the javascript.
> 
> 
> +1. Let's move the codebase forward, instead of continuing to build
> hacky workarounds for poor past architectural decisions.
> 
Coming from a distro perspective:

4.alt.2:
move to proper packages for static file management. I mean, they need to
be built anyways.

It has been mentioned, xstatic packages can block the gate. We currently
control xstatic package releases, thus we can roll back, if something
goes wrong.

If we're pulling directly with npm/bower, someone from the outside can
break us. We already have the situation with pypi packages.
With proper packages, we could even use the gate to release those
packages and thus verify, we're not breaking anyone.

Matthias

-- 
Matthias Runge 

Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham,
Michael O'Neill

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Tripp, Travis S]

> On 03/09/2016 04:43 PM, Serg Melikyan wrote:

>>>
>>> This is exactly my first thought. The plugins *must* depend on Horizon,
>>> and they have to use the same versions of XStatic packages anyways,
>
> >> so why specify the dependencies twice?
>>
>>
>> Plugins may require different xstatic library, which is not even used
>> by Horizon. Issue raised by Richard exists for plugins too, not only
>> for Horizon itself.
>
>
> How would such an xstatic library conflict with what is in Horizon then,
> though?

> Say there are two xstatic libraries, a and b. B depends on a. Horizon depends 
> only on a. Plugin-foo depends on b. Horizon updates to a version of a that 
> the version of b consumed by plugin-foo is not compatible with.

> Rob

Just FYI, this topic came up in the horizon IRC today at the 20:30:45 minute 
mark.:  
http://eavesdrop.openstack.org/meetings/horizon/2016/horizon.2016-03-09-20.00.log.html
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

Hi Serg,

There's a crucial difference: getting the solution to this wrong for
Horizon will break the gate for all OpenStack projects. Updating an xstatic
package that a plugin uses has no side-effect outside of the plugin.


  Richard

On 10 March 2016 at 02:43, Serg Melikyan  wrote:

> >This is exactly my first thought. The plugins *must* depend on Horizon,
> and they have to use the same versions of XStatic packages anyways, so why
> specify the dependencies twice?
>
> Plugins may require different xstatic library, which is not even used
> by Horizon. Issue raised by Richard exists for plugins too, not only
> for Horizon itself.
>
>
> On Wed, Mar 9, 2016 at 12:24 AM, Radomir Dopieralski
>  wrote:
> > On 03/08/2016 11:43 PM, David Lyle wrote:
> >
> >> I'm wondering if since horizon is really the only project consuming
> >> these xstatic libraries and none are likely to venture down this path of
> >> madness with us that it would be safe to manage the xstatic requirements
> >> and upper-constraints locally.
> >>
> >> Technically the plugins for horizon depend on this, but they depend via
> >> horizon. If they require specific versions that are not supported by
> >> Horizon, I think all bets are off anyway.
> >
> >
> > This is exactly my first thought. The plugins *must* depend on Horizon,
> > and they have to use the same versions of XStatic packages anyways, so
> > why specify the dependencies twice? If the changes between versions are
> > so big as to be breaking, then the plugins have to be updated to work
> > with the new Horizon anyways.
> >
> > --
> > Radomir Dopieralski
> >
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
> http://mirantis.com | smelik...@mirantis.com
>
> +1 (650) 440-8979
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Michael Krotscheck]

On Wed, Mar 9, 2016 at 7:58 AM Monty Taylor  wrote:

>
> > SOLUTION 4: vendor the javascript
> >
> > Heh.
>
> Heh indeed.
>
> SOLUTION 4.alt: use npm/bower instead of xstatic to pull the javascript.
>

+1. Let's move the codebase forward, instead of continuing to build hacky
workarounds for poor past architectural decisions.

Michael
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Robert Collins]

On 10 Mar 2016 2:58 AM, "Radomir Dopieralski" 
wrote:
>
> On 03/09/2016 04:43 PM, Serg Melikyan wrote:
>>>
>>> This is exactly my first thought. The plugins *must* depend on Horizon,
>>> and they have to use the same versions of XStatic packages anyways,
>
> >> so why specify the dependencies twice?
>>
>>
>> Plugins may require different xstatic library, which is not even used
>> by Horizon. Issue raised by Richard exists for plugins too, not only
>> for Horizon itself.
>
>
> How would such an xstatic library conflict with what is in Horizon then,
> though?

Say there are two xstatic libraries, a and b. B depends on a. Horizon
depends only on a. Plugin-foo depends on b. Horizon updates to a version of
a that the version of b consumed by plugin-foo is not compatible with.

Rob
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Radomir Dopieralski]

On 03/09/2016 04:43 PM, Serg Melikyan wrote:

This is exactly my first thought. The plugins *must* depend on Horizon,
and they have to use the same versions of XStatic packages anyways,

>> so why specify the dependencies twice?


Plugins may require different xstatic library, which is not even used
by Horizon. Issue raised by Richard exists for plugins too, not only
for Horizon itself.


How would such an xstatic library conflict with what is in Horizon then,
though?

--
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Monty Taylor]

On 03/07/2016 10:52 PM, Richard Jones wrote:

We've solved *most* of the issues around releasing new xstatic packages,
documented here[1] (and related documentation).

We have one final issue that's blocking us, which is that during the
xstatic release there will be a point at which Horizon may be broken
from an integrated point of view - some of the interfaces may not work
and fail tests. The process goes something like this:

​Note: this assumes that depends-on can reliably bring in patches from
all over the place into a gate environment, which is technically
possible, but not necessarily correct today.

The problem is that because we can't atomically update both
upper-constraints *and* Horizon at the same time (or upper-constraints
*and* xstatic-angular, or Horizon *and* xstatic-angular) we run into a
situation where Horizon will be running against the wrong version of
xstatic-angular.

So we break one of the basic assumptions of the OpenStack world: that
every single commit in every repository for the integrated environment
will pass tests.

In the Python world, we code around this by making Horizon compatible
with both the X and X1 versions of xstatic-angular (if it was a Python
library). In Javascript land this is much more difficult - Javascript
libraries tend to break compatibility in far more interesting ways.


Yah. Honestly, this is one of the main places where I think we get into 
trouble in linux-distro land in trying to apply the tools/techniques 
developed for one set of problems with another.



Maintaining compatibility across CSS and font releases is also a
difficult problem, though changes here are unlikely to break Horizon
enough that gate tests would fail. So, solution 1 to the problem is:

SOLUTION 1: always maintain Horizon compatibility across xstatic library
releases.

This is potentially very difficult to guarantee. So, a second solution
has been proposed:

SOLUTION 2: move the upper-constraints information for *the xstatic
libraries only* from the global upper-constraints file into Horizon's
repository.

This allows us to atomically update both Horizon and the xstatic library
version, removing the potential to break because of the version
mismatch. Unfortunately it means that we have version compatibility
issues with anything else that wants to install alongside Horizon that
also uses xstatic packages. For example, Horizon plugins. We could move
Horizon plugins into the Horizon repository to solve this. /ducks


I actually like this one a lot. I know that there is a plugin problem 
... but ... plugin installers could also consume the horizon xstatic 
constraints when installing xstatic packages?


xstatic packages are specifically workarounds for doing javascript in a 
python-centric world. Putting then in upper-constraints and actually 
treating them like actual python packages rather than what they are 
which is a clever utilization of the python ecosystem to deliver 
javascript libraries is vexing at best.




A variation on this solution is:

SOLUTION 3: allow Horizon to locally override upper-constraints for the
time needed to merge a patch in devstack.

This solution allows Horizon to atomically update itself and the xstatic
library, but it also means installing Horizon in a CI/CD environment
becomes more difficult due to the need to always pull down the
upper-constraints file and edit it. We could code this in to tox, but
that doesn't help eg. devstack which needs to also do this thing.


Or people who are deploying horizon CD from master from source and 
applying upper-constraints to get the tested version. Those people would 
have to duplicate the tox logic.



SOLUTION 4: vendor the javascript

Heh.


Heh indeed.

SOLUTION 4.alt: use npm/bower instead of xstatic to pull the javascript.



SOLUTION 5: have dependencies on xstatic move from global requirements
to Horizon

This is similar to 2 & 3 with some of the same drawbacks (multiple users
of xstatic) but also we'd need a change to global-requirements handling
to ignore some entries in Horizon's requirements.

Your thoughts on any and all of the above are requested.


Thanks for your work on this - it's a hard problem - especially with 
sets of potentially conflicting desires from your consumers.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Serg Melikyan]

>This is exactly my first thought. The plugins *must* depend on Horizon, and 
>they have to use the same versions of XStatic packages anyways, so why specify 
>the dependencies twice?

Plugins may require different xstatic library, which is not even used
by Horizon. Issue raised by Richard exists for plugins too, not only
for Horizon itself.


On Wed, Mar 9, 2016 at 12:24 AM, Radomir Dopieralski
 wrote:
> On 03/08/2016 11:43 PM, David Lyle wrote:
>
>> I'm wondering if since horizon is really the only project consuming
>> these xstatic libraries and none are likely to venture down this path of
>> madness with us that it would be safe to manage the xstatic requirements
>> and upper-constraints locally.
>>
>> Technically the plugins for horizon depend on this, but they depend via
>> horizon. If they require specific versions that are not supported by
>> Horizon, I think all bets are off anyway.
>
>
> This is exactly my first thought. The plugins *must* depend on Horizon,
> and they have to use the same versions of XStatic packages anyways, so
> why specify the dependencies twice? If the changes between versions are
> so big as to be breaking, then the plugins have to be updated to work
> with the new Horizon anyways.
>
> --
> Radomir Dopieralski
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
http://mirantis.com | smelik...@mirantis.com

+1 (650) 440-8979

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Radomir Dopieralski]

On 03/08/2016 11:43 PM, David Lyle wrote:


I'm wondering if since horizon is really the only project consuming
these xstatic libraries and none are likely to venture down this path of
madness with us that it would be safe to manage the xstatic requirements
and upper-constraints locally.

Technically the plugins for horizon depend on this, but they depend via
horizon. If they require specific versions that are not supported by
Horizon, I think all bets are off anyway.


This is exactly my first thought. The plugins *must* depend on Horizon,
and they have to use the same versions of XStatic packages anyways, so
why specify the dependencies twice? If the changes between versions are
so big as to be breaking, then the plugins have to be updated to work
with the new Horizon anyways.

--
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Robert Collins]

On 9 March 2016 at 14:23, Jeremy Stanley  wrote:
> On 2016-03-09 13:57:45 +1300 (+1300), Robert Collins wrote:
>> On 9 March 2016 at 13:07, Jeremy Stanley  wrote:
>> > On 2016-03-08 17:25:41 +1100 (+1100), Richard Jones wrote:
>> > [...]
>> >> SOLUTION 6 - make zuul capable of performing atomic cross-repository
>> >> commits.
>> >
>> > This seems unlikely to happen, as it's very much counter to Zuul's
>> > designed-in reliance on serializing changes to test before merging
>> > them.
>>
>> I'd like to explore this more, but a mailing list thread probably
>> isn't terribly efficient - I don't even know where to start to figure
>> out any differing assumptions, whether whats being propose is
>> conceptually desirable and-or how to represent that in zuul. But we've
>> spent nearly three days of bug smash here in sydney trying to get some
>> sort of design for going forward, and so far this is the only one that
>> hasn't bad pretty big negative external side effects such as 'break
>> every other user of xstatic when horizon updates'. :(.
>>
>> So -> IRC and/or/perhaps voice?
>
> ML thread will probably work for some initial exploration, but with
> James leading the Zuul v3 design and implementation he's in the best
> position to say whether it's going to be sane to support having
> changes in multiple repos which must test and merge together rather
> than being able to merge one before the other. Previously we've seen
> the idea of atomic merges coordinated across multiple repositories
> to be a sign of poor software design, and as such have actively
> discouraged the notion (it did come up briefly during the original
> cross-repo dependencies design, and was pretty quickly rejected as
> unsanitary).
>
> Anyway, I'll give him a heads up on this since he likely doesn't
> follow Horizon-specific discussions very closely.

Thanks. The driving factor here is that supporting both versions of
many of these static things - both JS, and the html that interacts
with it - is very tricky. E.g. having two copies of all templates
tricky.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Jeremy Stanley]

On 2016-03-09 13:57:45 +1300 (+1300), Robert Collins wrote:
> On 9 March 2016 at 13:07, Jeremy Stanley  wrote:
> > On 2016-03-08 17:25:41 +1100 (+1100), Richard Jones wrote:
> > [...]
> >> SOLUTION 6 - make zuul capable of performing atomic cross-repository
> >> commits.
> >
> > This seems unlikely to happen, as it's very much counter to Zuul's
> > designed-in reliance on serializing changes to test before merging
> > them.
> 
> I'd like to explore this more, but a mailing list thread probably
> isn't terribly efficient - I don't even know where to start to figure
> out any differing assumptions, whether whats being propose is
> conceptually desirable and-or how to represent that in zuul. But we've
> spent nearly three days of bug smash here in sydney trying to get some
> sort of design for going forward, and so far this is the only one that
> hasn't bad pretty big negative external side effects such as 'break
> every other user of xstatic when horizon updates'. :(.
> 
> So -> IRC and/or/perhaps voice?

ML thread will probably work for some initial exploration, but with
James leading the Zuul v3 design and implementation he's in the best
position to say whether it's going to be sane to support having
changes in multiple repos which must test and merge together rather
than being able to merge one before the other. Previously we've seen
the idea of atomic merges coordinated across multiple repositories
to be a sign of poor software design, and as such have actively
discouraged the notion (it did come up briefly during the original
cross-repo dependencies design, and was pretty quickly rejected as
unsanitary).

Anyway, I'll give him a heads up on this since he likely doesn't
follow Horizon-specific discussions very closely.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Robert Collins]

On 9 March 2016 at 13:07, Jeremy Stanley  wrote:
> On 2016-03-08 17:25:41 +1100 (+1100), Richard Jones wrote:
> [...]
>> SOLUTION 6 - make zuul capable of performing atomic cross-repository
>> commits.
>
> This seems unlikely to happen, as it's very much counter to Zuul's
> designed-in reliance on serializing changes to test before merging
> them.

I'd like to explore this more, but a mailing list thread probably
isn't terribly efficient - I don't even know where to start to figure
out any differing assumptions, whether whats being propose is
conceptually desirable and-or how to represent that in zuul. But we've
spent nearly three days of bug smash here in sydney trying to get some
sort of design for going forward, and so far this is the only one that
hasn't bad pretty big negative external side effects such as 'break
every other user of xstatic when horizon updates'. :(.

So -> IRC and/or/perhaps voice?

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

But Jeremy: Atomic Zuul. How cool is that name.


So I know I'm going to potentially get yelled at for voicing this, but what
are the chances that app-catalog and Horizon are ever installed on the same
system? That is, could it have its own xstatic constraints, with the
promise that the two constraints will never meet in a dark alley behind
some VM somewhere? app-catalog isn't in the integrated tests, so it's never
going to break the gate like Horizon will...


On 9 March 2016 at 11:07, Jeremy Stanley  wrote:

> On 2016-03-08 17:25:41 +1100 (+1100), Richard Jones wrote:
> [...]
> > SOLUTION 6 - make zuul capable of performing atomic cross-repository
> > commits.
>
> This seems unlikely to happen, as it's very much counter to Zuul's
> designed-in reliance on serializing changes to test before merging
> them.
> --
> Jeremy Stanley
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Jeremy Stanley]

On 2016-03-08 17:25:41 +1100 (+1100), Richard Jones wrote:
[...]
> SOLUTION 6 - make zuul capable of performing atomic cross-repository
> commits.

This seems unlikely to happen, as it's very much counter to Zuul's
designed-in reliance on serializing changes to test before merging
them.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[David Lyle]

On Mon, Mar 7, 2016 at 9:52 PM, Richard Jones 
wrote:

> We've solved *most* of the issues around releasing new xstatic packages,
> documented here[1] (and related documentation).
>
> We have one final issue that's blocking us, which is that during the
> xstatic release there will be a point at which Horizon may be broken from
> an integrated point of view - some of the interfaces may not work and fail
> tests. The process goes something like this:
>
> ​Note: this assumes that depends-on can reliably bring in patches from all
> over the place into a gate environment, which is technically possible, but
> not necessarily correct today.
>
> The problem is that because we can't atomically update both
> upper-constraints *and* Horizon at the same time (or upper-constraints
> *and* xstatic-angular, or Horizon *and* xstatic-angular) we run into a
> situation where Horizon will be running against the wrong version of
> xstatic-angular.
>
> So we break one of the basic assumptions of the OpenStack world: that
> every single commit in every repository for the integrated environment will
> pass tests.
>
> In the Python world, we code around this by making Horizon compatible with
> both the X and X1 versions of xstatic-angular (if it was a Python library).
> In Javascript land this is much more difficult - Javascript libraries tend
> to break compatibility in far more interesting ways. Maintaining
> compatibility across CSS and font releases is also a difficult problem,
> though changes here are unlikely to break Horizon enough that gate tests
> would fail. So, solution 1 to the problem is:
>
> SOLUTION 1: always maintain Horizon compatibility across xstatic library
> releases.
>
> This is potentially very difficult to guarantee. So, a second solution has
> been proposed:
>

This seems unlikely to go well.


>
> SOLUTION 2: move the upper-constraints information for *the xstatic
> libraries only* from the global upper-constraints file into Horizon's
> repository.
>
> This allows us to atomically update both Horizon and the xstatic library
> version, removing the potential to break because of the version mismatch.
> Unfortunately it means that we have version compatibility issues with
> anything else that wants to install alongside Horizon that also uses
> xstatic packages. For example, Horizon plugins. We could move Horizon
> plugins into the Horizon repository to solve this. /ducks
>

I don't know that you can duck low enough for that.

I'm wondering if since horizon is really the only project consuming these
xstatic libraries and none are likely to venture down this path of madness
with us that it would be safe to manage the xstatic requirements and
upper-constraints locally.

Technically the plugins for horizon depend on this, but they depend via
horizon. If they require specific versions that are not supported by
Horizon, I think all bets are off anyway.


>
> A variation on this solution is:
>
> SOLUTION 3: allow Horizon to locally override upper-constraints for the
> time needed to merge a patch in devstack.
>
> This solution allows Horizon to atomically update itself and the xstatic
> library, but it also means installing Horizon in a CI/CD environment
> becomes more difficult due to the need to always pull down the
> upper-constraints file and edit it. We could code this in to tox, but that
> doesn't help eg. devstack which needs to also do this thing.
>
>
I combined this with #2.


> SOLUTION 4: vendor the javascript
>
> Heh.
>

This makes me sad.


>
> SOLUTION 5: have dependencies on xstatic move from global requirements to
> Horizon
>
> This is similar to 2 & 3 with some of the same drawbacks (multiple users
> of xstatic) but also we'd need a change to global-requirements handling to
> ignore some entries in Horizon's requirements.
>
>
I guess I combined 2, 3 and 5. Just remove the xstatic overhead from the
openstack global mechanism.


> Your thoughts on any and all of the above are requested.
>
>
> Richard
>
> [1] https://review.openstack.org/#/c/289142/
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

Two things I forgot to mention:

Currently there is another break point in the diagram of releases when X1
is released. Currently Horizon does not use upper-constraints and thus will
immediately pick up the new xstatic release file, potentially breaking
things. This is easy to fix - I will be proposing a patch soon.

SOLUTION 6 - make zuul capable of performing atomic cross-repository
commits.

Richard

Sent from my portable device, please excuse the brevity.
On 8 Mar 2016 15:52, "Richard Jones"  wrote:

> We've solved *most* of the issues around releasing new xstatic packages,
> documented here[1] (and related documentation).
>
> We have one final issue that's blocking us, which is that during the
> xstatic release there will be a point at which Horizon may be broken from
> an integrated point of view - some of the interfaces may not work and fail
> tests. The process goes something like this:
>
> ​Note: this assumes that depends-on can reliably bring in patches from all
> over the place into a gate environment, which is technically possible, but
> not necessarily correct today.
>
> The problem is that because we can't atomically update both
> upper-constraints *and* Horizon at the same time (or upper-constraints
> *and* xstatic-angular, or Horizon *and* xstatic-angular) we run into a
> situation where Horizon will be running against the wrong version of
> xstatic-angular.
>
> So we break one of the basic assumptions of the OpenStack world: that
> every single commit in every repository for the integrated environment will
> pass tests.
>
> In the Python world, we code around this by making Horizon compatible with
> both the X and X1 versions of xstatic-angular (if it was a Python library).
> In Javascript land this is much more difficult - Javascript libraries tend
> to break compatibility in far more interesting ways. Maintaining
> compatibility across CSS and font releases is also a difficult problem,
> though changes here are unlikely to break Horizon enough that gate tests
> would fail. So, solution 1 to the problem is:
>
> SOLUTION 1: always maintain Horizon compatibility across xstatic library
> releases.
>
> This is potentially very difficult to guarantee. So, a second solution has
> been proposed:
>
> SOLUTION 2: move the upper-constraints information for *the xstatic
> libraries only* from the global upper-constraints file into Horizon's
> repository.
>
> This allows us to atomically update both Horizon and the xstatic library
> version, removing the potential to break because of the version mismatch.
> Unfortunately it means that we have version compatibility issues with
> anything else that wants to install alongside Horizon that also uses
> xstatic packages. For example, Horizon plugins. We could move Horizon
> plugins into the Horizon repository to solve this. /ducks
>
> A variation on this solution is:
>
> SOLUTION 3: allow Horizon to locally override upper-constraints for the
> time needed to merge a patch in devstack.
>
> This solution allows Horizon to atomically update itself and the xstatic
> library, but it also means installing Horizon in a CI/CD environment
> becomes more difficult due to the need to always pull down the
> upper-constraints file and edit it. We could code this in to tox, but that
> doesn't help eg. devstack which needs to also do this thing.
>
> SOLUTION 4: vendor the javascript
>
> Heh.
>
> SOLUTION 5: have dependencies on xstatic move from global requirements to
> Horizon
>
> This is similar to 2 & 3 with some of the same drawbacks (multiple users
> of xstatic) but also we'd need a change to global-requirements handling to
> ignore some entries in Horizon's requirements.
>
> Your thoughts on any and all of the above are requested.
>
>
> Richard
>
> [1] https://review.openstack.org/#/c/289142/
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

We've solved *most* of the issues around releasing new xstatic packages,
documented here[1] (and related documentation).

We have one final issue that's blocking us, which is that during the
xstatic release there will be a point at which Horizon may be broken from
an integrated point of view - some of the interfaces may not work and fail
tests. The process goes something like this:

​Note: this assumes that depends-on can reliably bring in patches from all
over the place into a gate environment, which is technically possible, but
not necessarily correct today.

The problem is that because we can't atomically update both
upper-constraints *and* Horizon at the same time (or upper-constraints
*and* xstatic-angular, or Horizon *and* xstatic-angular) we run into a
situation where Horizon will be running against the wrong version of
xstatic-angular.

So we break one of the basic assumptions of the OpenStack world: that every
single commit in every repository for the integrated environment will pass
tests.

In the Python world, we code around this by making Horizon compatible with
both the X and X1 versions of xstatic-angular (if it was a Python library).
In Javascript land this is much more difficult - Javascript libraries tend
to break compatibility in far more interesting ways. Maintaining
compatibility across CSS and font releases is also a difficult problem,
though changes here are unlikely to break Horizon enough that gate tests
would fail. So, solution 1 to the problem is:

SOLUTION 1: always maintain Horizon compatibility across xstatic library
releases.

This is potentially very difficult to guarantee. So, a second solution has
been proposed:

SOLUTION 2: move the upper-constraints information for *the xstatic
libraries only* from the global upper-constraints file into Horizon's
repository.

This allows us to atomically update both Horizon and the xstatic library
version, removing the potential to break because of the version mismatch.
Unfortunately it means that we have version compatibility issues with
anything else that wants to install alongside Horizon that also uses
xstatic packages. For example, Horizon plugins. We could move Horizon
plugins into the Horizon repository to solve this. /ducks

A variation on this solution is:

SOLUTION 3: allow Horizon to locally override upper-constraints for the
time needed to merge a patch in devstack.

This solution allows Horizon to atomically update itself and the xstatic
library, but it also means installing Horizon in a CI/CD environment
becomes more difficult due to the need to always pull down the
upper-constraints file and edit it. We could code this in to tox, but that
doesn't help eg. devstack which needs to also do this thing.

SOLUTION 4: vendor the javascript

Heh.

SOLUTION 5: have dependencies on xstatic move from global requirements to
Horizon

This is similar to 2 & 3 with some of the same drawbacks (multiple users of
xstatic) but also we'd need a change to global-requirements handling to
ignore some entries in Horizon's requirements.

Your thoughts on any and all of the above are requested.


Richard

[1] https://review.openstack.org/#/c/289142/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev