[openstack-dev] [Horizon] Some thoughts about Horizon's test suite

[Richard Jones]

I'm relatively new to Horizon, but I come with a bunch of experience with
Python more generally. I've contributed a couple of small patches to
Horizon in an effort to get more familiar with the codebase. Julie Pichon's
blog post about testing in Horizon has been invaluable 
http://www.jpichon.net/blog/2013/07/testing-in-horizon/.

Very recently I attempted to fix a simple bug in which a Panel was being
displayed when it shouldn't have been. The resultant 5-line fix ended up
breaking 498 of the 1048 unit tests in the suite. I estimated that it would
take about a week's effort to address all the failing tests. For more
information see 
http://mechanicalcat.net/richard/log/Python/When_testing_goes_bad

In talking about the issues I've hit I found quite a few people both in
#openstack-horizon and other places who are quite forthright in their
desire to get rid of all use of mox. I heartily endorse this motion.
Indeed, I would be willing to start the process of weeding mox out of the
suite entirely, one test module at a time, if there is support for such an
effort from Horizon core/TL.

Removing mox is one (admittedly very large) piece of the puzzle though. As
I mentioned in my blog post, there are other structural issues that would
need to be addressed also; the use of unittest.TestCase and the varying
methods of mocking.

At the moment, making a fix like the one I attempted is prohibitively
expensive to do, and thus won't be done because Horizon's test suite has
become too restrictive to change (unnecessarily so). I'd like to see that
change.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Some thoughts about Horizon's test suite

[Richard Jones]

Thanks for your thoughts Radomir. The nova api in question is memoized so
it'll only be called once per request. Caching it for longer would be a
very good idea, but that then brings into play deeper knowledge than I have
about how long to cache things like nova extension configuration. Also, I
looked into this see whether we could use a nicer existing memoizing system
(one with a timeout, that doesn't use weakref and will clean out stale
entries), but none of them will handle the existence of the varying request
parameter, so more work would be required to build our own solution. It's
still something I'd like to see done.

But that's not really the point of this email, as you note :)



On 29 August 2014 19:46, Radomir Dopieralski openst...@sheep.art.pl wrote:

 On 29/08/14 04:22, Richard Jones wrote:


  Very recently I attempted to fix a simple bug in which a Panel was being
  displayed when it shouldn't have been. The resultant 5-line fix ended up
  breaking 498 of the 1048 unit tests in the suite. I estimated that it
  would take about a week's effort to address all the failing tests. For
  more information see
  http://mechanicalcat.net/richard/log/Python/When_testing_goes_bad

 Having read that, I can't help but comment that maybe, just maybe,
 making an API call on each an every request to Horizon is not such a
 great idea after all, and should be very well thought out, as it is
 costly. In particular, it should be investigated if the call could be
 made only on some requests? That would have the side effect of breaking
 much fewer tests.

 But I agree that mox is horrible in that it effectively freezes the
 implementation details of the tested unit, instead of testing its behavior.

 --
 Radomir Dopieralski


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Supporting Javascript clients calling OpenStack APIs

[Richard Jones]

[This is Horizon-related but affects every service in OpenStack, hence no
filter in the subject]

I would like for OpenStack to support browser-based Javascript API clients.
Currently this is not possible because of cross-origin resource blocking in
Javascript clients - that is, given some Javascript hosted on
https://horizon.company.com/; you cannot, for example, call from that
Javascript code to an API on https://apis.company.com:5000/v2.0/tokens; to
authenticate with Keystone.

There are three solutions to this problem:

1. the Horizon solution, in which those APIs are proxied by a very thick
   layer of additional Python API, plus some Python view code with some
   Javascript on the top only calling the Horizon view code,
2. add CORS support to all the OpenStack APIs though a new WSGI middleware
   (for example oslo.middleware.cors) and configured into each of the API
   services individually since they all exist on different origin
   host:port combinations, or
3. a new web service that proxies all the APIs and serves the static
   Javascript (etc) content from the one origin (host). APIs are then served
   from new URL roots /name/ where the name is from the serviceCatalog
   entry. Static content can be served from /static/. The serviceCatalog
from
   keystone will be rewritten on the fly to point the API publicURLs at the
   new service. Requests are no longer cross-origin.

I have implemented options 2 and 3 as an exercise to see how horrid each one
is.


== CORS Middleware ==

For those wanting a bit of background, I have written up a spec for oslo
that
talks about how this could work: https://review.openstack.org/#/c/119485/

The middleware option results in a reasonably nice bit of middleware. It's
short and relatively easy to test. The big problem with it comes in
configuring it in all the APIs. The configuration for the middleware takes
two forms:

1. hooking oslo.middleware.cors into the WSGI pipeline (there's more than
   one in each API),
2. adding the CORS configuration itself for the middleware in the API's main
   configuration file (eg. keystone.conf or nova.conf).

So for each service, that's two configuration files *and* the kicker is that
the paste configuration file is non-trivially different in almost every
case.

That's a lot of work, and confusing for deployers. Configuration management
tools can ease *some* of this burden (the *.conf files) but those paste
files
are a bit of a mess :(

Once the config change is in place, it works (well, except for an issue I
ran
into relating to oslo.middleware.sizelimit which I'll go into in another
place).

The implementation hasn't been pushed up for review as I'm not sure it
should
be. I can do this if people wish me to.


== New Single-Point API Service ==

Actually, this is not horrid in any way - unless that publicURL rewriting
gives you the heebie-jeebies.

It works, and offers us some nice additional features like being able to
host
the service behind SSL without needing to get a bazillion certificates. And
maybe load balancing. And maybe API access filtering.

I note that https://openrepose.org already exists to be *something* like
this, but it's not *precisely* what I'm proposing. Also Java euwww ;)


So, I propose that the idea of CORS-in-all-the-things as an idea be
put aside as unworkable.

I intend to pursue the single-point API service that I have described as a
way of moving forward in prototyping a pure-Javascript OpenStack Dashboard.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Supporting Javascript clients calling OpenStack APIs

[Richard Jones]

On 12 September 2014 07:50, Adam Young ayo...@redhat.com wrote:

  On 09/11/2014 03:15 AM, Richard Jones wrote:

  [This is Horizon-related but affects every service in OpenStack, hence no
 filter in the subject]

  I would like for OpenStack to support browser-based Javascript API
 clients.
 Currently this is not possible because of cross-origin resource blocking in
 Javascript clients - that is, given some Javascript hosted on
 https://horizon.company.com/; you cannot, for example, call from that
 Javascript code to an API on https://apis.company.com:5000/v2.0/tokens;
 to
 authenticate with Keystone.

  There are three solutions to this problem:

  1. the Horizon solution, in which those APIs are proxied by a very thick
layer of additional Python API, plus some Python view code with some
Javascript on the top only calling the Horizon view code,
 2. add CORS support to all the OpenStack APIs though a new WSGI middleware
(for example oslo.middleware.cors) and configured into each of the API
services individually since they all exist on different origin
host:port combinations, or
 3. a new web service that proxies all the APIs and serves the static
Javascript (etc) content from the one origin (host). APIs are then
 served
from new URL roots /name/ where the name is from the serviceCatalog
entry. Static content can be served from /static/. The serviceCatalog
 from
keystone will be rewritten on the fly to point the API publicURLs at the
new service. Requests are no longer cross-origin.

  I have implemented options 2 and 3 as an exercise to see how horrid each
 one
  is.


 I don't think these are mutually exclusive.  I can see people wanting
 either in some deployments.


I think I agree :)


== CORS Middleware ==

  For those wanting a bit of background, I have written up a spec for oslo
 that
 talks about how this could work: https://review.openstack.org/#/c/119485/

  The middleware option results in a reasonably nice bit of middleware.
 It's
 short and relatively easy to test. The big problem with it comes in
 configuring it in all the APIs. The configuration for the middleware takes
 two forms:

  1. hooking oslo.middleware.cors into the WSGI pipeline (there's more than
one in each API),
 2. adding the CORS configuration itself for the middleware in the API's
 main
configuration file (eg. keystone.conf or nova.conf).

  So for each service, that's two configuration files *and* the kicker is
 that
 the paste configuration file is non-trivially different in almost every
 case.

 This is one reason I thought that it should be done by auth_token
 middleware.  The other reason is that I don't think we want to blanket
 accept CORS from everywhere, but instead we should do so based on the
 service catalog.


It's very important to understand that CORS is entirely advisory. Nothing
is required to pay any attention to it. Modern browsers do, of course, but
in the absence of a browser initiating a CORS conversation (by sending an
Origin header) the CORS middleware should do nothing whatsoever. A GET
request will still return the body of data requested - it's just that the
browser will see the CORS header and block an application from seeing that
data. Security through access controls, XSS protections, etc. must be
handled by other mechanisms.


For a POC deployment, for a small company, all-in-one,  what you are doing
 shouild be fine, but then, if you were running all of your services that
 way, in one web server, you wouldn't need CORS either.


This isn't possible in the current OpenStack model - all the APIs run on
different ports, which count as different origins for cross-origin resource
issues (origin is defined as {protocol, host, port}).


So, lets have these two approaches work in parallel.  THe proxy will get
 things goint while we work out the CORS approach.


I will look at submitting my middleware for inclusion anyway then.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Supporting Javascript clients calling OpenStack APIs

[Richard Jones]

On 12 September 2014 09:24, Richard Jones r1chardj0...@gmail.com wrote:

 On 12 September 2014 07:50, Adam Young ayo...@redhat.com wrote:

 So, lets have these two approaches work in parallel.  THe proxy will get
 things goint while we work out the CORS approach.


 I will look at submitting my middleware for inclusion anyway then.


Submitted to oslo.middleware https://review.openstack.org/#/c/120964/


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon]Dev environment default login

[Richard Jones]

I needs to point at a devstack or openstack installation for login to work.

On 12 September 2014 15:50, Rajdeep Dua rajdeep@gmail.com wrote:

 Hi,
 I have setup a local dev environment with a custom dashboard using
 instructions below

 http://docs.openstack.org/developer/horizon/topics/tutorial.html

 Horizon started using ./run_tests.sh --runserver 0.0.0.0:8877
 What is the default password for admin login?

 Note : it is not pointing to any openstack installation yet

 Thanks
 Rajdeep



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] OpenStack bootstrapping hour - Friday Sept 19th - 3pm EST

[Richard Jones]

This is a great idea, thanks!

On 16 September 2014 08:56, Sean Dague s...@dague.net wrote:

 A few of us have decided to pull together a regular (cadence to be
 determined) video series taking on deep dives inside of OpenStack,
 looking at code, explaining why things work that way, and fielding
 questions from anyone interested.

 For lack of a better title, I've declared it OpenStack Bootstrapping Hour.

 Episode 0 - Mock best practices will kick off this Friday, Sept 19th,
 from 3pm - 4pm EST. Our experts for this will be Jay Pipes and Dan
 Smith. It will be done as a Google Hangout on Air, which means there
 will be a live youtube stream while it's on, and a recorded youtube
 video that's publicly accessible once we're done.

 We'll be using an etherpad during the broadcast to provide links to the
 content people are looking at, as well as capture questions. That will
 be our backchannel, and audience participation forum, with the advantage
 that it creates a nice concise document at the end of the broadcast that
 pairs well with the video. (Also: the tech test showed that while code
 examples are perfectly viewable during in the final video, during the
 live stream they are a little hard to read, etherpad links will help
 people follow along at home).

 Assuming this turns out to be useful, we're thinking about lots of other
 deep dives. The intent is that these are indepth dives. We as a
 community have learned so many things over the last 4 years, but as
 OpenStack has gotten so large, being familiar with more than a narrow
 slice is hard. This is hopefully a part of the solution to address that.
 As I've told others, if nothing else, I'm looking forward to learning a
 ton in the process.

 Final links for the hangout + etherpad will be posted a little later in
 the week. Mostly wanted to make people aware it was coming.

 -Sean

 --
 Sean Dague
 http://dague.net

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone][Horizon] CORS and Federation

[Richard Jones]

You're quite probably correct - going through the OWASP threat list in more
detail is on my TODO. That was just off the top of my head as something
that has me concerned but I've not investigated it thoroughly.

On 17 September 2014 14:15, Adam Young ayo...@redhat.com wrote:

  On 09/16/2014 08:56 PM, Richard Jones wrote:

 CORS for all of OpenStack is possible once the oslo middleware lands*, but
 as you note it's only one of many elements to be considered when exposing
 the APIs to browsers. There is no current support for CSRF protection in
 the OpenStack APIs, for example. I believe that sort of functionality
 belongs in an intermediary between the APIs and the browser.


 Typically, CSRF is done by writing a customer header.  Why wouldn't the
 -X-Auth-Token header qualify?  Its not a cookie, automatically added.  So,
 CORS support would be necesary for horizon to send the token on a request
 to Nova, but no other site would be able to do that.  No?




  Richard

  * https://review.openstack.org/#/c/120964/

 On 17 September 2014 08:59, Gabriel Hurley gabriel.hur...@nebula.com
 wrote:

 This is generally the right plan. The hard parts are in getting people to
 deploy it correctly and securely, and handling fallback cases for lack of
 browser support, etc.

 What we really don't want to do is to encourage people to set
 Access-Control-Allow-Origin: * type headers or other such nonsense simply
 because it's too much work to do things correctly. This becomes especially
 challenging for federated clouds.

 I would encourage looking at the problem of adding all the necessary
 headers for CORS as an OpenStack-wide issue. Once you figure it out for
 Keystone, the next logical step is to want to make calls from the browser
 directly to all the other service endpoints, and each service is going to
 have to respond with the correct CORS headers
 (Access-Control-Allow-Methods and Access-Control-Allow-Headers are
 particularly fun ones for projects like Glance or Swift). A common
 middleware and means of configuring it will go a long way to easing user
 pain and spurring adoption of the new mechanisms. It will help the Horizon
 team substantially in the long run to do it consistently and predictably
 across the stack.

 As a side-note, once we're in the realm of handling all this sensitive
 data with the browser as a middleman, encouraging people to configure
 things like CSP is probably also a good idea to make sure we're not loading
 malicious scripts or other resources.

 Securing a browser-centric world is a tricky realm... let's make sure we
 get it right. :-)

  - Gabriel

  -Original Message-
  From: Adam Young [mailto:ayo...@redhat.com]
  Sent: Tuesday, September 16, 2014 3:40 PM
  To: OpenStack Development Mailing List
  Subject: [openstack-dev] [Keystone][Horizon] CORS and Federation
 
  Phase one for dealing with Federation can be done with CORS support
 solely
  for Keystone/Horizon  integration:
 
  1.  Horizon Login page creates Javascript to do AJAX call to Keystone 2.
  Keystone generates a token 3.  Javascript reads token out of response
 and
  sends it to Horizon.
 
  This should support Kerberos, X509, and Password auth;  the Keystone
 team
  is discussing how to advertise mechanisms, lets leave the onus on us to
 solve
  that one and get back in a timely manner.
 
  For Federation, the handshake is a little more complex, and there might
 be a
  need for some sort of popup window for the user to log in to their home
  SAML provider.  Its several more AJAX calls, but the end effect should
 be the
  same:  get a standard Keystone token and hand it to Horizon.
 
  This would mean that Horizon would have to validate tokens the same way
  as any other endpoint.  That should not be too hard, but there is a
 little bit of
  create a user, get a token, make a call logic that currently lives
 only in
  keystonemiddleware/auth_token;  Its a solvable problem.
 
  This approach will support the straight Javascript approach that
 Richard Jones
  discussed;  Keystone behind a proxy will work this way without CORS
  support.  If CORS  can be sorted out for the other services, we can do
 straight
  Javascript without the Proxy.  I see it as phased approach with this
 being the
  first phase.
 
 
 
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 ___
 OpenStack-dev mailing 
 listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman

Re: [openstack-dev] [TripleO] pypi-mirror is now unsupported - what do we do now?

[Richard Jones]

On 10 July 2014 02:19, Ben Nemec openst...@nemebean.com wrote:

 On 07/08/2014 11:05 PM, Joe Gordon wrote:
  On Tue, Jul 8, 2014 at 8:54 PM, James Polley j...@jamezpolley.com wrote:
 
  It may not have been clear from the below email, but clarkb clarifies on
  https://bugs.launchpad.net/openstack-ci/+bug/1294381 that the infra
 team
  is no longer maintaining pypi-mirror
 
  This has been a very useful tool for tripleo. It's much simpler for new
  developers to set up and use than a full bandersnatch mirror (and
 requires
  less disk space), and it can create a local cache of wheels which saves
  build time.
 
  But it's now unsupported.
 
  To me it seems like we have two options:
 
  A) Deprecate usage of pypi-mirror; update docs to instruct new devs in
  setting up a local bandersnatch mirror instead
  or
  B) Take on care-and-feeding of the tool.
  or, I guess,
  C) Continue to recommend people use an unsupported unmaintained
  known-buggy tool (it works reasonably well for us today, but it's going
 to
  work less and less well as time goes by)
 
  Are there other options I haven't thought of?
 
 
  I don't know if this fits your requirements but I use
  http://doc.devpi.net/latest/quickstart-pypimirror.html for my
 development
  needs.

 Will that also cache wheels?  In my experience, wheels are one of the
 big time savers in tripleo so I would consider it an important feature
 to maintain, however we decide to proceed.


Yes, devpi caches wheels.

I would suggest that if the pip cache approach isn't appropriate then devpi
probably a good solution (though I don't know your full requirements).

The big difference between using devpi and pip caching would be that devpi
will allow you to install packages when you're offline.


   Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] pypi-mirror is now unsupported - what do we do now?

[Richard Jones]

On 10 July 2014 23:27, Mulcahy, Stephen stephen.mulc...@hp.com wrote:
 When I last tested bandersnatch, it didn’t work well behind a proxy (in
fact most of the existing pypi mirroring tools suffered from the same
problem) – pypi-mirror has worked extremely well for mirroring a subset of
pypi and doing so behind a proxy. I’d also echo the requirement for a tool
that provides wheels as we have seen significant performance improvement
from using wheels with TripleO

devpi works behind a proxy. If bandersnatch doesn't then that bug should be
addressed ASAP. I'm in contact with its author regarding that.

I'm currently investigating a sensible approach to having wheels be
automatically built (for the most sensible value of automatic that we can
determine wink).


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] concrete proposal on changing the library testing model with devstack-gate

[Richard Jones]

On 30 September 2014 08:14, Doug Hellmann d...@doughellmann.com wrote:


 On Sep 29, 2014, at 4:22 PM, Robert Collins robe...@robertcollins.net
 wrote:

  As far as pip goes, you may not know, but tox defaults to pip --pre,
  which means anyone using tox, like us all here, will be pulling the
  alphas down by default: so impacting folk doing e.g. devtest on
  icehouse. So I don't think the hack is working as intended.

 I dislike tox more every day. Is this why we have the installation command
 override set in tox.ini?


Yes. This issue in the tox tracker has some background - feel free to
upvote it!

https://bitbucket.org/hpk42/tox/issue/193/remove-the-pre-pip-option-by-default


Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

Hi all,

At the summit last week, we developed a plan for moving forward with
modernising Horizon's UI using AngularJS. If you weren't at that meeting
and are interested in helping out with this effort please let me know!

The relevant etherpad from the meeting:
https://etherpad.openstack.org/p/kilo-horizon-contributors-meetup

TL;DR: piece by piece we will replace Django views in Horizon with angular
views, and we're going to start with Identity

First up, I'd like to ask the UX folk who raised their hands in that
meeting to indicate which of the Identity panes we should start with. I
believe a wizard was mentioned, as a way to exercise the new wizard code
from Maxime.

At the same time, I'm looking at updating the AngularJS recommendations in
the wiki. I believe other aspects of the current approach to angular code
should also be revisited, if we're to scale up to the full angular
front-end envisaged. I'd appreciate if those interested in this aspect in
particular could contact me so we can sort this out as a team!

I'd like to start the design work for the new REST API layer we'll be
exposing to the angular application code, but that is also part of the
broader discussion about the structure of the angular code in the Horizon
application as mentioned above. Should it be a new blueprint/spec?

There were some discussions around tooling. We're using xstatic to manage
3rd party components, but there's a lot missing from that environment. I
hesitate to add supporting xstatic components on to the already large pile
of work we have to do, so would recommend we switch to managing those
components with bower instead. For reference the list of 3rd party
components I used in angboard* (which is really only a teensy fraction of
the total application we'd end up with, so this components list is probably
reduced):

json3
es5-shim
angular
angular-route
angular-cookies
angular-animate
angular-sanitize
angular-smart-table
angular-local-storage
angular-bootstrap
angular-translate
font-awesome
boot
underscore
ng-websocket

Just looking at PyPI, it looks like only a few of those are in xstatic, and
those are out of date.

grunt provides a lot of features for developing an angular interface. In
particular LiveReload accelerates development significantly. There's a
django-livereload but it uses tiny-lr under the hood, so we're still using
a node application for LiveReload support... so it might make sense to just
use grunt. grunt provides many other features as well (wiredep integration
with bower, build facilities with ngMin, test monitoring and reload etc).

There seemed to be agreement to move to jasmine (from qunit) for writing
the tests. It's not noted in the etherpad, but I recall karma was accepted
as a given for the test runner. For those not in the meeting, angboard uses
mocha+chai for test writing, but I agreed that jasmine is acceptable, and
is already used by Storyboard (see below).

Also, phantomjs so we don't have to fire up a browser for exercising (what
should hopefully be an extensive) unit test suite.

The Storyboard project has successfully integrated these tools into the
OpenStack CI environment.


 Richard

* https://github.com/r1chardj0n3s/angboard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] proposal: alternating weekly meeting time

[Richard Jones]

Hi all,

At the summit meetup last week I proposed that the Horizon weekly meeting
time alternate between the current time and something more suitable for
those of us closer to UTC+10. I'd like to get an indication of the interest
in this, and I'll look into getting a second meeting time booked for
alternating weeks based on your feedback.

As a starting point, I'd like to suggest the times alternate between UTC
 and 1600 (the current time).


Thanks,

Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 11 November 2014 20:10, Anton Zemlyanov azemlya...@mirantis.com wrote:

 Modernizing Horizon using Angular is really great. I would suggest a
 couple of minor things to consider.

 First, font-awesome is excellent and free font, but there are two
 problems: it has lots of icons that are not used, it also miss some other
 icons worth to have. I would suggest Fontello, it is tool for picking
 glyphs from all the free fonts round there and building custom fonts


Thanks for bringing this up. We discussed the need for additional symbols
at the meetup, and it may very well be that we do come up with a custom
font with a selection from font-awesome plus additional symbols designed as
we go along. Very good idea.



 Grunt is being forced out by Gulp that is times faster while having the
 same level of plugins and tools and is easier to use


Thanks for the pointer. I mentioned grunt in particular because it's quite
mature at this point, was the default tool for the yoeman project builder I
used and is also the tool of choice for Storyboard. If there is a
compelling reason to switch though, I imagine the Storyboard folk may also
be interested. From a brief look around there does seem to be a lot of
enthusiasm for gulp. I will look at making a branch of angboard that uses
gulp to get a good feeling for it (unless someone produces a PR that does
it for me ;)


 Richard


 On Tue, Nov 11, 2014 at 11:02 AM, Richard Jones r1chardj0...@gmail.com
 wrote:

 Hi all,

 At the summit last week, we developed a plan for moving forward with
 modernising Horizon's UI using AngularJS. If you weren't at that meeting
 and are interested in helping out with this effort please let me know!

 The relevant etherpad from the meeting:
 https://etherpad.openstack.org/p/kilo-horizon-contributors-meetup

 TL;DR: piece by piece we will replace Django views in Horizon with
 angular views, and we're going to start with Identity

 First up, I'd like to ask the UX folk who raised their hands in that
 meeting to indicate which of the Identity panes we should start with. I
 believe a wizard was mentioned, as a way to exercise the new wizard code
 from Maxime.

 At the same time, I'm looking at updating the AngularJS recommendations
 in the wiki. I believe other aspects of the current approach to angular
 code should also be revisited, if we're to scale up to the full angular
 front-end envisaged. I'd appreciate if those interested in this aspect in
 particular could contact me so we can sort this out as a team!

 I'd like to start the design work for the new REST API layer we'll be
 exposing to the angular application code, but that is also part of the
 broader discussion about the structure of the angular code in the Horizon
 application as mentioned above. Should it be a new blueprint/spec?

 There were some discussions around tooling. We're using xstatic to manage
 3rd party components, but there's a lot missing from that environment. I
 hesitate to add supporting xstatic components on to the already large pile
 of work we have to do, so would recommend we switch to managing those
 components with bower instead. For reference the list of 3rd party
 components I used in angboard* (which is really only a teensy fraction of
 the total application we'd end up with, so this components list is probably
 reduced):

 json3
 es5-shim
 angular
 angular-route
 angular-cookies
 angular-animate
 angular-sanitize
 angular-smart-table
 angular-local-storage
 angular-bootstrap
 angular-translate
 font-awesome
 boot
 underscore
 ng-websocket

 Just looking at PyPI, it looks like only a few of those are in xstatic,
 and those are out of date.

 grunt provides a lot of features for developing an angular interface. In
 particular LiveReload accelerates development significantly. There's a
 django-livereload but it uses tiny-lr under the hood, so we're still using
 a node application for LiveReload support... so it might make sense to just
 use grunt. grunt provides many other features as well (wiredep integration
 with bower, build facilities with ngMin, test monitoring and reload etc).

 There seemed to be agreement to move to jasmine (from qunit) for writing
 the tests. It's not noted in the etherpad, but I recall karma was accepted
 as a given for the test runner. For those not in the meeting, angboard uses
 mocha+chai for test writing, but I agreed that jasmine is acceptable, and
 is already used by Storyboard (see below).

 Also, phantomjs so we don't have to fire up a browser for exercising
 (what should hopefully be an extensive) unit test suite.

 The Storyboard project has successfully integrated these tools into the
 OpenStack CI environment.


  Richard

 * https://github.com/r1chardj0n3s/angboard

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 11 November 2014 20:53, Jiri Tomasek jtoma...@redhat.com wrote:

  Hey,

 Thanks for writing this up!

 I am including some notes and questions inline...

 On 11/11/2014 08:02 AM, Richard Jones wrote:

 Hi all,

  At the summit last week, we developed a plan for moving forward with
 modernising Horizon's UI using AngularJS. If you weren't at that meeting
 and are interested in helping out with this effort please let me know!

  The relevant etherpad from the meeting:
 https://etherpad.openstack.org/p/kilo-horizon-contributors-meetup

  TL;DR: piece by piece we will replace Django views in Horizon with
 angular views, and we're going to start with Identity

  First up, I'd like to ask the UX folk who raised their hands in that
 meeting to indicate which of the Identity panes we should start with. I
 believe a wizard was mentioned, as a way to exercise the new wizard code
 from Maxime.

  At the same time, I'm looking at updating the AngularJS recommendations
 in the wiki. I believe other aspects of the current approach to angular
 code should also be revisited, if we're to scale up to the full angular
 front-end envisaged. I'd appreciate if those interested in this aspect in
 particular could contact me so we can sort this out as a team!

 I am interested.


Excellent!



 I'd like to start the design work for the new REST API layer we'll be
 exposing to the angular application code, but that is also part of the
 broader discussion about the structure of the angular code in the Horizon
 application as mentioned above. Should it be a new blueprint/spec?


 I think spec seems appropriate.  Do you think using django-angular would
 be convenient?


As I understand it, django-angular exists to make it easier to integrate
Django views and angular views, whereas what we're after is for them to be
as separate as possible, with Django providing a single templated page that
forms the base page for the angular single-page app, and the REST API is as
un-Django as possible (using something like tastypie).




 There were some discussions around tooling. We're using xstatic to manage
 3rd party components, but there's a lot missing from that environment. I
 hesitate to add supporting xstatic components on to the already large pile
 of work we have to do, so would recommend we switch to managing those
 components with bower instead. For reference the list of 3rd party
 components I used in angboard* (which is really only a teensy fraction of
 the total application we'd end up with, so this components list is probably
 reduced):

  json3
 es5-shim
 angular
 angular-route
 angular-cookies
 angular-animate
 angular-sanitize
 angular-smart-table
  angular-local-storage
 angular-bootstrap
 angular-translate
 font-awesome
 boot
 underscore
 ng-websocket

  Just looking at PyPI, it looks like only a few of those are in xstatic,
 and those are out of date.

  grunt provides a lot of features for developing an angular interface. In
 particular LiveReload accelerates development significantly. There's a
 django-livereload but it uses tiny-lr under the hood, so we're still using
 a node application for LiveReload support... so it might make sense to just
 use grunt. grunt provides many other features as well (wiredep integration
 with bower, build facilities with ngMin, test monitoring and reload etc).

  There seemed to be agreement to move to jasmine (from qunit) for writing
 the tests. It's not noted in the etherpad, but I recall karma was accepted
 as a given for the test runner. For those not in the meeting, angboard uses
 mocha+chai for test writing, but I agreed that jasmine is acceptable, and
 is already used by Storyboard (see below).

  Also, phantomjs so we don't have to fire up a browser for exercising
 (what should hopefully be an extensive) unit test suite.

  The Storyboard project has successfully integrated these tools into the
 OpenStack CI environment.


 Using javascript tooling (yeoman, grunt, bower, etc.) has this issue of
 being dependent on nodejs which if I recall correctly is causing problems
 for packagers as some versions of these tools require different nodejs
 versions - please Mathias correct me if I am wrong. I know this discussion
 has been here before, but using these tools is necessary for effective
 development. So we need to resolve the problem asap. Storyboard does not
 have this issue as it is infra thing.


Infra still did have to do work to make things reliably build, and part of
their solution was to install a pinned version of node.js using nodeenv -
that is, node is installed into and alongside the virtualenv just like
Python packages are. Using a pinned version removes the version
compatibility issue, and using nodeenv means there's no impact to the
broader system like you'd get with a standard node install.



Petr Belanyi has added optional jshint install for js linting into Horizon
 and it installs nodejs as it depends

Re: [openstack-dev] URLs

[Richard Jones]

On 12 November 2014 09:47, Ian Cordasco ian.corda...@rackspace.com wrote:

 On 11/11/14, 16:35, Adam Young ayo...@redhat.com wrote:

 Recent recurrence of the Why ios everything on its own port question
 triggered my desire to take this pattern and put it to rest.
 
 My suggestion, from a while ago, was to have a naming scheme that
 deconflicts putting all of the services onto a single server, on port 443.
 
 I've removed a lot of the cruft, but not added in entries for all the
 new *aaS services.
 
 
 https://wiki.openstack.org/wiki/URLs
 
 Please add in anything that should be part of OpenStack.  Let's make
 this a reality, and remove the  specific ports.


What do those URLs map to? For example, what does
https://hostname/identity/main map to, and does this scheme intend to
handle multiple endpoints per service?



 Is the reason subdomains aren’t being suggested because of the expense of
 *.hostname certificates?


Well, it's also much more effort even without certs :)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

I have set up a doodle poll to let folk enter their preferred times. It's
in UTC/GMT (/London time, because doodle) so use something like
http://everytimezone.com/ to figure that out :)

https://doodle.com/47h3f35nad62ncnf


 Richard

On 11 November 2014 18:46, Matthias Runge mru...@redhat.com wrote:

 On 11/11/14 08:09, Richard Jones wrote:
  Hi all,
 
  At the summit meetup last week I proposed that the Horizon weekly
  meeting time alternate between the current time and something more
  suitable for those of us closer to UTC+10. I'd like to get an indication
  of the interest in this, and I'll look into getting a second meeting
  time booked for alternating weeks based on your feedback.
 
  As a starting point, I'd like to suggest the times alternate between UTC
   and 1600 (the current time).

 Sadly, both times don't work for me. I would propose something like 8
 UTC, which should work for most folks located in EU and east, or 18 UTC.

 Matthias


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 12 November 2014 18:17, Matthias Runge mru...@redhat.com wrote:

 On 11/11/14 10:53, Jiri Tomasek wrote:
  Hey,
 
  Thanks for writing this up!

  The Storyboard project has successfully integrated these tools into
  the OpenStack CI environment.

 OpenStack CI and distributors are different, because OpenStack CI does
 not distribute software.


Ah, I wasn't clear; my concern was whether the tools chosen would be
compatible with the CI environment. I'm hoping that distribution of the
tools isn't our concern (see below).



  Using javascript tooling (yeoman, grunt, bower, etc.) has this issue of
  being dependent on nodejs which if I recall correctly is causing
  problems for packagers as some versions of these tools require different
  nodejs versions - please Mathias correct me if I am wrong. I know this
  discussion has been here before, but using these tools is necessary for
  effective development. So we need to resolve the problem asap.
  Storyboard does not have this issue as it is infra thing.

 As far as I know, those tools don't require different nodejs versions.
 But: we can not have different node.js versions installed at the same
 time. I assume, this is true for all distributions. Creating and
 maintaining parallel installable versions just sucks and causes many
 issues.


I believe the nodeenv method of installing node solves this, as it's
entirely local to the development environment.


I will have to go through all dependencies and do a review, if those are
 acceptable for inclusion e.g in Fedora. The same is true for Thomas
 Goirand for inclusion in Debian.

 
  Petr Belanyi has added optional jshint install for js linting into
  Horizon and it installs nodejs as it depends on it. Could this approach
  work for our need of js tooling too? [1]

 Sigh, this nonsense doesn't go away? This is the third time the same
 issue comes up.

 jshint is NOT free software.

 https://github.com/jshint/jshint/blob/master/src/jshint.js#L19
 https://github.com/jshint/jshint/blob/master/src/jshint.js#L19


They're trying to resolve that https://github.com/jshint/jshint/issues/1234

But regardless, jshint doesn't have to be installed from a Linux
repository; it's usually installed using npm alongside the other node tools.


Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

To clarify - I created the doodle in UTC (so the times in there are UTC
00:00 Monday to 23:59 Friday) but they will be shifted to your local
timezone - scroll all the way to the right to double-check that it's set
correctly for you.

Sorry for the confusion.


 Richard

On 12 November 2014 12:45, Richard Jones r1chardj0...@gmail.com wrote:

 I have set up a doodle poll to let folk enter their preferred times. It's
 in UTC/GMT (/London time, because doodle) so use something like
 http://everytimezone.com/ to figure that out :)

 https://doodle.com/47h3f35nad62ncnf


  Richard

 On 11 November 2014 18:46, Matthias Runge mru...@redhat.com wrote:

 On 11/11/14 08:09, Richard Jones wrote:
  Hi all,
 
  At the summit meetup last week I proposed that the Horizon weekly
  meeting time alternate between the current time and something more
  suitable for those of us closer to UTC+10. I'd like to get an indication
  of the interest in this, and I'll look into getting a second meeting
  time booked for alternating weeks based on your feedback.
 
  As a starting point, I'd like to suggest the times alternate between UTC
   and 1600 (the current time).

 Sadly, both times don't work for me. I would propose something like 8
 UTC, which should work for most folks located in EU and east, or 18 UTC.

 Matthias


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 13 November 2014 09:35, Adam Young ayo...@redhat.com wrote:

 On 11/12/2014 03:03 AM, Matthias Runge wrote:

 On 12/11/14 08:40, Richard Jones wrote:

  I believe the nodeenv method of installing node solves this, as it's
 entirely local to the development environment.

 See below, this touches package build as well.


  I will have to go through all dependencies and do a review, if
 those are
  acceptable for inclusion e.g in Fedora. The same is true for Thomas
  Goirand for inclusion in Debian.

  
   Petr Belanyi has added optional jshint install for js linting into
   Horizon and it installs nodejs as it depends on it. Could this
 approach
   work for our need of js tooling too? [1]

  Sigh, this nonsense doesn't go away? This is the third time the same
  issue comes up.

  jshint is NOT free software.

  https://github.com/jshint/jshint/blob/master/src/jshint.js#L19


 They're trying to resolve that https://github.com/jshint/
 jshint/issues/1234

 But regardless, jshint doesn't have to be installed from a Linux
 repository; it's usually installed using npm alongside the other node
 tools.

  Thanks for the pointer, this is good news!

 Regarding package managers, my POV is completely different. From a
 distributor perspective, where customers expect everything provided from
 a single source, I'm not using npm, pip etc. I'm packaging that stuff
 properly before using it.

 There are companies out there, where security policy does not allow to
 install software from a third party repository. pypi etc. are considered
 as a third party here in this context.

 I would prefer to have the complete tool chain available as (RPM)
 packages. I am executing unit tests etc. during package build. Our
 builders don't have access to the internet, downloading any other stuff
 from the internet is no option.


 And this is really not-negotiable, too.  Debian has the same requirements,
 it is not Red Hat/Fedora speciufic.  It is no different than the Python
 Code.  We dodn't pip install for RHOSP for the Python packages, and we
 can't use any of the language specific package managers.

 But, we are used to it, and dealing with package dependencies is what we
 do.  Having these in Fedora, while painful, will be ultimately very
 valuable.


I guess a point of difference here is that the Linux packaging systems
target global installation, whereas bower and npm (in our case) target a
local installation. It's effectively vendoring the code, but you still have
to pull it down from a repository for each installation. Don't worry, I'm
not going to suggest we actually move back to vendoring :)

We're currently using xstatic and that works with Linux packaging because
it was designed to cope with being a global installation. The current
Horizon codebase has a django-xstatic plugin which further makes dealing
with xstatic components nicer - for example it handles path management and
static file compilation (JS minification and concatenation, for example).
That's really nice, but poses some problems:

- we would need to xstatic-ify (and deb/rpm-ify) all those components
- we could run into global version conflict issues if we run more than one
service on a system - is this likely to be an issue in practise though?
- as far as I'm aware, the xstatic JS minification is not angular-aware,
and will break angular code that has not been written to be
dumb-minifier-aware (the angular minifier ngMin is written in node and
knows how to do things more correctly); adding dumb-minifier-awareness to
angular code makes it ugly and more error-prone :/

On the minification issue though: I talked to a few people at the summit
and we agreed that minification should just be avoided; gzip at the
transport layer buys significantly better compression, and has the added
bonus that the resultant code is readable. I note that the Debian JS
guidelines* only recommend that libraries *should* be minified (though I'm
unsure why they even recommend that).

One further note on xstatic vs. bower: during development it's very useful
to be able to install random components from bower to try them out; during
angboard development I would sometimes install and remove half a dozen
components just in one day. I would hope that whatever solution we come up
with has at least some of this flexibility for experimentation.


Richard

* https://wiki.debian.org/Javascript/Policy
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

Gabriel has responded saying very much what I would have said, so I won't
repeat that. I would like to note though that bower and npm are two
separate beasts entirely. The dependency trees in bower are very limited
indeed (only two additional components are installed beyond the list below)
which is in stark contrast to npm. The xstatic question only applies to the
bower components - the npm stuff is all tool chain; tools that I believe
should be packaged as such by packagers.

I would like to address one specific point Thomas raised though:

On 13 November 2014 12:29, Thomas Goirand z...@debian.org wrote:

 I just don't understand why all of this is needed. I did some JS
 programming myself back in the days (10 years ago, using PHP...). I
 could do Ajax by hand, without using a single library. What is it that
 you're doing in Horizon that needs so many libs? When I see Horizon in
 Juno, I don't even get it. Or is it because you just want to have fancy
 animation? Frankly, I don't care these. I care a lot more that we're not
 adding new potential security problems.


Horizon is an incredibly complex application. Just so we're all on the same
page, the components installed by bower for angboard are:

angular
  Because writing an application the size of Horizon without it would be
madness :)
angular-route
  Provides structure to the application through URL routing.
angular-cookies
  Provides management of browser cookies in a way that integrates well with
angular.
angular-sanitize
  Allows direct embedding of HTML into angular templates, with sanitization.
json3
  Compatibility for older browsers so JSON works.
es5-shim
  Compatibility for older browsers so Javascript (ECMAScript 5) works.
angular-smart-table
  Table management (population, sorting, filtering, pagination, etc)
angular-local-storage
   Browser local storage with cookie fallback, integrated with angular
mechanisms.
angular-bootstrap
   Extensions to angular that leverage bootstrap (modal popups, tabbed
displays, ...)
font-awesome
   Additional glyphs to use in the user interface (warning symbol, info
symbol, ...)
boot
   Bootstrap for CSS styling (this is the dependency that brings in jquery
and requirejs)
underscore
   Javascript utility library providing a ton of features Javascript lacks
but Python programmers expect.
ng-websocket
   Angular-friendly interface to using websockets
angular-translate
   Support for localization in angular using message catalogs generated by
gettext/transifex.
angular-mocks
   Mocking support for unit testing angular code
angular-scenario
   More support for angular unit tests

Additionally, angboard vendors term.js because it was very poorly packaged
in the bower ecosystem. +1 for xstatic there I guess :)

So those are the components we needed to create the prototype in a few
weeks. Not using them would have added months (or possibly years) to the
development time. Creating an application of the scale of Horizon without
leveraging all that existing work would be like developing OpenStack while
barring all use of Python 3rd-party packages.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 14 November 2014 02:04, Thomas Goirand z...@debian.org wrote:

 On 11/13/2014 12:13 PM, Richard Jones wrote:
  the npm stuff is all tool chain; tools
  that I believe should be packaged as such by packagers.

 npm is already in Debian:
 https://packages.debian.org/sid/npm

 However, just like we can't use CPAN, pear install, pip install and
 such when building or installing package, we wont be able to use NPM.
 This means every single dependency that isn't in Debian will need to be
 packaged.


Just to be clearer, when I wrote the npm stuff I meant npm and the tools
installed by it, so grunt, bower, karma, phantomjs, etc. Not the stuff
managed by bower, just the stuff installed by npm. Those npm-based things
should be packaged by the distros as tools, just like other programs the
distros package.


 Horizon is an incredibly complex application. Just so we're all on the
  same page, the components installed by bower for angboard are:
 
  angular
Because writing an application the size of Horizon without it would be
  madness :)
  angular-route
Provides structure to the application through URL routing.
  angular-cookies
Provides management of browser cookies in a way that integrates well
  with angular.
  angular-sanitize
Allows direct embedding of HTML into angular templates, with
 sanitization.
  json3
Compatibility for older browsers so JSON works.
  es5-shim
Compatibility for older browsers so Javascript (ECMAScript 5) works.
  angular-smart-table
Table management (population, sorting, filtering, pagination, etc)
  angular-local-storage
 Browser local storage with cookie fallback, integrated with angular
  mechanisms.
  angular-bootstrap
 Extensions to angular that leverage bootstrap (modal popups, tabbed
  displays, ...)
  font-awesome
 Additional glyphs to use in the user interface (warning symbol, info
  symbol, ...)
  boot
 Bootstrap for CSS styling (this is the dependency that brings in
  jquery and requirejs)
  underscore
 Javascript utility library providing a ton of features Javascript
  lacks but Python programmers expect.
  ng-websocket
 Angular-friendly interface to using websockets
  angular-translate
 Support for localization in angular using message catalogs generated
  by gettext/transifex.
  angular-mocks
 Mocking support for unit testing angular code
  angular-scenario
 More support for angular unit tests
 
  Additionally, angboard vendors term.js because it was very poorly
  packaged in the bower ecosystem. +1 for xstatic there I guess :)
 
  So those are the components we needed to create the prototype in a few
  weeks. Not using them would have added months (or possibly years) to the
  development time. Creating an application of the scale of Horizon
  without leveraging all that existing work would be like developing
  OpenStack while barring all use of Python 3rd-party packages.

 I have no problem with adding dependencies. That's how things work, for
 sure, I just want to make sure it doesn't become hell, with so many
 components inter-depending on 100s of them, which would become not
 manageable. If we define clear boundaries, then fine! The above seems
 reasonable anyway.

 Though did you list the dependencies of the above?


Again, just so we're clear, yes, the above is *all* the components
installed by bower, including dependencies (jquery and requirejs being the
*only* dependencies not directly installed).

As I mentioned, the dependency trees in bower are significantly simpler
than npm packages tend to be (most bower packages have zero or one
dependency). The 100s of dependencies are in npm packages - but as Martin
Gleiser has pointed out, npm solves that problem with its local install and
node_modules directory structures.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

I think that it boils down to whether it'is possible that distributions:

1. package the node-based tools (grunt, karma, protractor, ...) as
installable programs, and
2. xstatic-package the bower-based packages that we use (probably a couple
of dozen at least).

We might even be able to get away without using grunt, though an
alternative to its LiveReload facility (and one that doesn't then also
depend on another node program like django-livereload does) would be
required. I believe tox and django's runserver (and other manage commands)
could suffice to replace the other functionality typically provided by
grunt.


Richard

On 14 November 2014 18:51, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 13/11/14 23:30, Martin Geisler wrote:

 [...]

  While I agree that it's chaotic, I also think you make the problem worse
  than it really is. First, remember that the user who installs Horizon
  won't need to use the JavaScript based *developer* tools such as npm,
  bower, etc.
 
  That is, I think Horizon developers will use these tools to produce a
  release -- a tarball -- and that tarball will be something you unpack on
  your webserver and then you're done. I base this on what I've seen in
  the project I've been working. The release tarball you download here
  don't mention npm, bower, or any of the other tools:
 
https://github.com/zerovm/swift-browser/releases
 
  The tools were used to produce the tarball and were used to test it, but
  they're not part of the released product. Somewhat similar to how GCC
  isn't included in the tarball if you download a pre-compiled binary.

 [...]

  Maybe a difference is that you don't (yet) install a web application
  like you install a system application. Instead you *deploy* it: you
  unpack files on a webserver, you configure permissions, you setup cache
  rules, you configure a database, etc.

 [...]

 I think I see where the misunderstanding is in this whole discussion. It
 seems it revolves around the purpose and role of the distribution.

 From the naive point of view, the role of a Linux distribution is to
 just collect all the software from respective upstream developers and
 put it in a single repository, so that it can be easily installed by the
 users. That's not the case.

 The role of a distribution is to provide a working ecosystem of
 software, that is:
 a) installed and configured in consistent way,
 b) tested to work with the specific versions that it ships with,
 c) audited for security,
 d) maintained, including security patches,
 e) documented, including external tutorials and the like,
 f) supported, either by the community or by the companies that provide
 support,
 g) free of licensing issues and legal risks as much as possible,
 h) managed with the common system management tools.

 In order to do that, they can't just take a tarball and drop it in a
 directory. They always produce their own builds, to make sure it's the
 same thing that the source code specifies. They sometimes have to
 rearrange configuration files, to make them fit the standards in their
 system. They provide sane configuration defaults. They track the
 security reports about all the installed components, and apply fixes,
 often before the security issue is even announced.

 Basically, a distribution adds a whole bunch of additional guarantees
 for the software they ship. Those are often long-term guarantees, as
 they will be supporting our software long after we have forgotten about
 it already.

 You say that web development doesn't work like that. That may be true,
 but that's mostly because if you develop a new web application in-house,
 and deploy it on your server, you don't really have such large legal
 risk, configuration complexity or support problem -- you just have to
 care about that single application, because the packagers from the
 distribution that you are using are taking care about all the rest of
 software on your server.

 --
 Radomir Dopieralski


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

I did it that way so that the form wouldn't blow out completely, and I
thought for our purposes of scheduling a 2 hour precision would be good
enough.

On 14 November 2014 22:32, Jaromir Coufal jcou...@redhat.com wrote:

 On 12/11/14 02:45, Richard Jones wrote:

 I have set up a doodle poll to let folk enter their preferred times.
 It's in UTC/GMT (/London time, because doodle) so use something like
 http://everytimezone.com/ to figure that out :)

 https://doodle.com/47h3f35nad62ncnf


   Richard


 Quick Question:

 Why is the length of the time slot 2h? Since it should take just 1h,
 should I consider beginning or end of the meeting to be relevant? It is
 challenging to find time slot between other meetings, since I might have
 just 1h slot there.

 -- Jarda


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 15 November 2014 00:58, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 14/11/14 13:02, Richard Jones wrote:
  I think that it boils down to whether it'is possible that
  distributions:
 
  1. package the node-based tools (grunt, karma, protractor, ...) as
  installable programs, and
  2. xstatic-package the bower-based packages that we use (probably a
  couple of dozen at least).
 
  We might even be able to get away without using grunt, though an
  alternative to its LiveReload facility (and one that doesn't then also
  depend on another node program like django-livereload does) would be
  required. I believe tox and django's runserver (and other manage
  commands) could suffice to replace the other functionality typically
  provided by grunt.

 We don't really need Xstatic for that. The packagers can as well package
 the bower-based packages directly. We can use anything, really,
 as long as we follow a process that makes sure that Horizon can be
 packaged into the different distributions. That is, we need:


xstatic provides additional meta-data that makes it much easier to
integrate the static bundle into an application - specifically it
automatically provides the application with file locations and endpoints
needed to be inserted into the application HTML. That stuff would have to
be done manually without xstatic, and would be a configuration pain.



 1. All dependencies explicit (with tests failing if a dependency is
missing),


xstatic provides us with a dependency mechanism through standard Python
setuptools facilities.



 2. explicit version ranges,


xstatic is done through requirements.txt, so yep :)



 3. no multiple versions of the same library,


This is not a thing in bower/client-side land. It's really only an issue
for npm-based packages, and as I've mentioned, the things we're using there
should be packaged as tools by the linux vendor, rather than accessed as
packages by our system. Except on non-Linux, of course, where we'll just
use npm ;)

So I guess the big open question is about how distros are going to deal
with npm tool packaging. I can't really answer that question for them
(except to say: don't try to replace npm's dependency solution with your
own; it simply won't work
because you'll probably never find a set of versions that satisfy even just
the few tools we're looking at for this project).


4. additions and upgrades of libraries moderated by the packagers,


Is there already some mechanism for handling the creation and management of
xstatic packages and how they interact with linux packagers? Sorry if this
is a noob question.



 5. ability to replace the development environment with packaged
libraries from the system,


I would very much like to still use bower for rapid development and
testing, with xstatic coming in once the experimentation is over. But if
there was a tool to automatically generate an xstatic package from a bower
one, that would be eaiser...



 6. ability to build and test our software with the tools that can be
used by all the distributions.


Yep, I think that just implies that the xstatic stuff is done, and that the
distros package the few npm tools we use.


On 15 November 2014 01:05, Thomas Goirand z...@debian.org wrote:

 On 11/11/2014 03:02 PM, Richard Jones wrote:
  json3
  es5-shim
  angular
  angular-route
  angular-cookies
  angular-animate
  angular-sanitize
  angular-smart-table
  angular-local-storage
  angular-bootstrap
  angular-translate
  font-awesome
  boot
  underscore
  ng-websocket

 Just FYI, in Debian, the libjs-angularjs already carries:

 angular-route.js
 angular-cookies.js
 angular-animate.js
 angular-sanitize.js

 We also already have packaged:
 font-awesome
 underscore

 So, basically, I'd have to package:
 json3
 es5-shim
 boot
 angular-smart-table
 angular-local-storage
 angular-translate
 ng-websocket

 That's a reasonable amount of work. Multiply this by 2 for the xstatic
 packages (if we keep using that), that's about 14 new packages.


The issue is integration with the Django application. How do we know what
the file path is to the entry point for that that code, and also how it
differs from other packagers? xstatic takes care of that for us (in much
the same way that bower does), so is valuable.



 By the way, can't we use libjs-sockjs instead of ng-websocket?


They offer different functionality, as far as I can tell. sockjs is a
compatibility layer thing, I think. I'm not sure. The description is a
little vague. On the other hand, ng-websocket is all about providing an
interface for angular applications over the top of websockets (and a handy
mock).



 Last, I'm ok if we add all these, but please, let's do this in the
 beginning of the Kilo cycle. It was really hard to cope with it at the
 end of the freeze for Juno.


I'd imagine this should be reasonable, barring any additional dependencies
we decide to include along the way

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 17 November 2014 06:42, Thomas Goirand z...@debian.org wrote:

 On 11/15/2014 06:27 AM, Gabriel Hurley wrote:
  So, I propose that a group gets together and defines criteria:
  we need to accept that the Horizon team (and those knowledgeable
  about web-app development) know best what tools they need, and
  they need to produce such a list as a starting point. We then
  need packagers and maintainers to examine that list and evaluate
  it for problems (non-free software, irresolvable dependencies,
  etc.). They're looking strictly for things which are un-packageable,
  not commenting on the necessity of said software. And we need people
  (thanks, Monty) willing to build new tools to find a way to turn
  that list into something the package maintainers can consume
  without burdening either side.
 
  Make sense?
 
   - Gabriel

 I'd be happy to be in this group.

As would I, hence I started the conversation in the first place :)


 Selenium
 
 As I wrote previously, the biggest issue currently for me, is selenium.
 It is very frustrating that I can't run these unit tests when building
 package, and potentially loose the opportunity to discover problems. I
 really would like this to be solved. There's 2 ways of solving it:

 1/ Someone works so that the .xpi can be built together with the rest of
 selenium, and therefore, selenium can leave the non-free repository of
 Debian and go in main (and also be uploaded in other distros).

 2/ We move away from Selenium and decide to use something else like
 PhantomJS.

I think you're confusing a couple of things here. Selenium is a system for
Javascript running tests in a browser environment. To do that, it needs a
browser. PhantomJS provides a headless browser to do that. The tests end up
being faster, less intrusive on a desktop and much simpler to run on
servers (no virtual X11 shenanigans).

I advocate for using PhantomJS, but also for using a reduced Selenium suite
where possible - with an emphasis on unit testing of the angular code
directly. Selenium is just so flaky, especially with an interface that's
even slightly dynamic.


 What we care is to find a system that will satisfy both worlds:
 distributions  upstream fast moving development. It is looking like NPM
 has the best feature and that it would be a winner against Bower and
Grunt.

Again, just to be clear: npm and bower are *both* packaging systems and
have completely separate domains of use. npm is used to package
command-line tools and libraries written in the node.js programming
language whereas bower is used to package browser application components.
It's not npm-or-bower.

bower is most likely going to be replaced by xstatic in our environment,
assuming we have some simple mechanism for creating xstatic packages from
bower components.

The distributions are going to have to package up the npm-packaged tools
that we need, though that set of packages is looking smaller and smaller,
and will probably just include the testing tools (karma, protractor,
jasmin, phantomjs). Some of those are already packaged \o/


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 17 November 2014 11:11, Thomas Goirand z...@debian.org wrote:

 On 11/15/2014 05:03 AM, Matthias Runge wrote:
  On 14/11/14 16:21, Adam Young wrote:
 
  Example:  I don't need Grunt to run a web server.  I need Apache for
  that.  Grunt does not need to be in the distro, mod_wsgi does.
 
  I will need every tool required to run e.g. unit tests or selenium tests
  to be packaged. Why? Because our builders don't have access to the
  internet and I want to be sure, the packaged version of horizon still
  passes tests.
 
  Matthias

 Except that selenium is non-free: it's in the non-free repository of
 Debian, because it contains a pre-built .xpi plugin for firefox, which
 itself contains pre-built .so and .dll files.


Hasn't this issue already been addressed? Horizon currently uses
Selenium-based integration tests.



 So, we're on the same page, but selenium is a bad tool... :)

 Chatting with Maxime Vidori on IRC, there's a bunch of alternative
 solutions that we could use instead of Selenium. PhantomJS for example
 is already in Debian main. I would very much welcome switching to that
 instead of Selenium.


PhantomJS isn't a testing framework - it's just a headless browser that's
scriptable from Javascript. It has a webdriver for Selenium, which is a
testing framework. If you ditch Selenium then you need to replace it with a
testing framework on top of PhantomJS.

There's also CasperJS which is an entirely separate testing framework built
over PhantomJS (and SlimerJS, which looks to be similar to PhantomJS except
built on Gecko - but not headless). It's very JUnit. Some people like that
;)

While Selenium has issues, I'm concerned that there's very few people out
there in the wilds doing interface testing without it. Indeed there's
people advocating for Selenium over PhantomJS directly. Using PhantomJS
directly means we won't have a suite of tests we can then target at IE,
Safari, etc to ensure that the interface works there in an automated
fashion. PhantomJS might be built on WebKit, but it's different enough from
Chrome that you still want to run your tests on Chrome to ensure the
interface actually works. SlimerJS does allow the testing to be slightly
broader, but we'd still miss automated IE and Safari tests.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 17 November 2014 21:54, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 17/11/14 09:53, Martin Geisler wrote:

 [...]

  As Richard said, npm and bower are not competitors. You use npm to
  install bower, and you use bower to download Angular, jQuery, Bootstrap
  and other static files. These are the static files that you will want to
  include when you finally deploy the web app to your server.
 
  Before using Bower, people would simply download Angular from the
  projects homepage and check it into version control. Bower is not doing
  much, but using it avoids this bad practice.
 
  There is often a kind of compilation step between bower downloading a
  dependency and the deployment on the webserver: minification and
  compression of the JavaScript and CSS. Concatenating and minifying the
  files serve to reduce the number of HTTP requests -- which can make an
  app much faster.
 
  Finally, you use Grunt/Gulp to execute other tools during development.
  These tools could be a local web server, it could be running the unit
  tests. Grunt is only a convenience tool here -- think of it as a kind of
  Makefile that tells you how to lunch various tasks.

 Thank you for your explanations.

 The way I see it, we would need:
 - Grunt both in the development environment and packaged (to run tests,
 etc.),

I'm increasingly coming to think that we can avoid grunt.

- serve is handled by django runserver
- test running is handled by tox
- livereload could be handled by https://pypi.python.org/pypi/livereload
(it'd be really nice if someone could get support for this rolled in to
django-livereload...)
- watch is not handled by anything and it would be a shame to miss out on
automatic linting/testing, but I think we can live without it

So, the tools we need packaged for Linux are:

- karma
- jasmine (already in Fedora, I believe)
- selenium (I believe this is already done in Fedora and Debian)
- phantomjs (definitely already done!)


 - Bower in the development environment,
 - Bower configuration file in two copies, one for global-requirements,
 and one for the Horizon's local requirements. Plus a gate job that makes
 sure no new library or version gets included in the Horizon's before
 getting into the global-requirements,

Could you perhaps elaborate on this? How do you see the workflow working
here?

Given that Horizon already integrates with xstatic, it would be messy (and
potentially confusing) to include something so it *also* integrated with
bower. I was envisaging us creating a tool which generates xstatic packages
from bower packages. I'm not the first to think along these lines
http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

I will be looking into creating such a tool today.


 - A tool, probably a script, that would help packaging the Bower
 packages into DEB/RPM packages. I suspect the Debian/Fedora packagers
 already have a semi-automatic solution for that.

Yep, that is indeed their problem that they'd have already solved for the
existing xstatic packages.


 - A script that would generate a file with all the paths to those
 packaged libraries, that would get included in Horizon's settings.py

If we stick with xstatic, we don't need this :)


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 18 November 2014 10:59, Richard Jones r1chardj0...@gmail.com wrote:

 On 17 November 2014 21:54, Radomir Dopieralski openst...@sheep.art.pl
 wrote:
  - Bower in the development environment,
  - Bower configuration file in two copies, one for global-requirements,
  and one for the Horizon's local requirements. Plus a gate job that makes
  sure no new library or version gets included in the Horizon's before
  getting into the global-requirements,

 Could you perhaps elaborate on this? How do you see the workflow working
 here?

 Given that Horizon already integrates with xstatic, it would be messy (and
 potentially confusing) to include something so it *also* integrated with
 bower. I was envisaging us creating a tool which generates xstatic packages
 from bower packages. I'm not the first to think along these lines
 http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

 I will be looking into creating such a tool today.


I wrote the tool today, and you can find it here:

https://github.com/r1chardj0n3s/flaming-shame

(github auto-named the repository for me - it's like it KNOWS)

I've proposed to Thomas Waldmann that this be included in the xstatic
package.

It doesn't handle dependencies at all - I'm not sure it should or could
sensibly.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 18 November 2014 19:22, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 18/11/14 00:59, Richard Jones wrote:
  On 17 November 2014 21:54, Radomir Dopieralski openst...@sheep.art.pl
  mailto:openst...@sheep.art.pl wrote:
  - Bower in the development environment,
  - Bower configuration file in two copies, one for global-requirements,
  and one for the Horizon's local requirements. Plus a gate job that makes
  sure no new library or version gets included in the Horizon's before
  getting into the global-requirements,
 
  Could you perhaps elaborate on this? How do you see the workflow working
  here?

 Basically we would have an additional file in the global-requirements
 directory, for listing the JavaScript dependencies. Then we would have a
 check on the Horizon's gate that would check Horizon's Bower
 configuration against that global-requirements file.

 This way we keep the same process for JavaScript libraries as we have
 for Python libraries: first you submit a patch to the
 global-requirements and have the dependency accepted by the infra team
 and packagers (checked against licenses, version conflicts, etc.), and
 then you add it to Horizon's dependencies. Of course you can submit both
 patches at once, just the Horizon one will fail the gate until the
 global-requirements one gets merged.

  Given that Horizon already integrates with xstatic, it would be messy
  (and potentially confusing) to include something so it *also* integrated
  with bower. I was envisaging us creating a tool which generates xstatic
  packages from bower packages. I'm not the first to think along these
  lines
 http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

 If we use Bower, we don't need to use Xstatic. It would be pure
 overhead. Bower already takes care of tracking releases and versions,
 and of bundling the files. All we need is a simple line in the
 settings.py telling Django where it puts all the files -- we don't
 really need Xstatic just for that. The packagers can then take those
 Bower packages and turn them into system packages, and just add/change
 the paths in settings.py to where they put the files. All in one place.


I guess I got the message that turning bower packages into system packages
was something that the Linux packagers were not keen on. Did I get the
message wrong there? If so, *and* we can get the bower stuff through #infra
and global-requirements then yes, we should totally try to avoid adding the
xstatic layer :)


Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 19 November 2014 03:14, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 18/11/14 12:54, Matthias Runge wrote:
  On Tue, Nov 18, 2014 at 09:36:00PM +1100, Richard Jones wrote:
  I guess I got the message that turning bower packages into system
 packages
  was something that the Linux packagers were not keen on. Did I get the
  message wrong there? If so, *and* we can get the bower stuff through
 #infra
  and global-requirements then yes, we should totally try to avoid adding
 the
  xstatic layer :)
 
  For me, it's more work to turn packages into bower, or to translate
  bower packages to system packages.
 
  But that is purely because we don't have bower packaged currently in
 Fedora.
  I would vote for the cleaner way (whatever that is).
 
  XStatic is obviously not the cleanest way, but a good compromise in most
  situations.

 The way I thought about it, we would simply replace the Bower packages
 with the corresponding system packages, by just changing the path in the
 settings.py file.

 Maybe an example would help illustrate it.
 Currently we have something like this:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular).base_dir),
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular_cookies).base_dir),
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular_mock).base_dir),
 ('horizon/lib/bootstrap_datepicker',
xstatic.main.XStatic(xstatic.pkg.bootstrap_datepicker).base_dir),
 ('bootstrap',
xstatic.main.XStatic(xstatic.pkg.bootstrap_scss).base_dir),
 ...
 ]

 We would replace that with:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
os.path.join(BASE_DIR, 'bower_modules/angular'),
 ...
 ]

 or wherever bower puts the files. Now, the packagers would replace those
 lines with:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
'/usr/lib/javascript/angular'),
 ...
 ]

 or whatever the packages in their distribution use. It's less work than
 having to make a whole xstatic package just to put that single path in
 there. The horizon system package would already depend on all the
 javascript library packages, so we are sure the files are there.
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


+1 to all that, except I'd recommend using django-bower to handle the
static collection stuff. It's not documented but django-bower has a setting
BOWER_COMPONENTS_ROOT which would make the above transition much simpler.
You leave it alone for local dev, and packagers setup BOWER_COMPONENTS_ROOT
to '/usr/lib/javascript/' or wherever.

bower also has a concept of entry points - there a main value in the
bower.json which identifies the Javascript, CSS, font and other files to
include in the index.html to have the library loaded into your application.
Saves a bunch of manual editing to get things right when you include the
library. Sadly, the current django-bower plugin doesn't use any of that,
though it does handle the collectstatic stuff.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

Thanks everyone who has responded. I'm going to leave the poll open until
the weekend to allow for stragglers to get their times in, and then close
it and we can see what the results are.

The poll is at https://doodle.com/47h3f35nad62ncnf scroll to the far right
to set your timezone.


 Richard

On 12 November 2014 12:45, Richard Jones r1chardj0...@gmail.com wrote:

 I have set up a doodle poll to let folk enter their preferred times. It's
 in UTC/GMT (/London time, because doodle) so use something like
 http://everytimezone.com/ to figure that out :)

 https://doodle.com/47h3f35nad62ncnf


  Richard

 On 11 November 2014 18:46, Matthias Runge mru...@redhat.com wrote:

 On 11/11/14 08:09, Richard Jones wrote:
  Hi all,
 
  At the summit meetup last week I proposed that the Horizon weekly
  meeting time alternate between the current time and something more
  suitable for those of us closer to UTC+10. I'd like to get an indication
  of the interest in this, and I'll look into getting a second meeting
  time booked for alternating weeks based on your feedback.
 
  As a starting point, I'd like to suggest the times alternate between UTC
   and 1600 (the current time).

 Sadly, both times don't work for me. I would propose something like 8
 UTC, which should work for most folks located in EU and east, or 18 UTC.

 Matthias


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

I've just had a long discussion with #infra folk about the
global-requirements thing, which deviated (quite naturally) into a
discussion about packaging (and their thoughts were in line with where
Radomir and I are heading).

In their view, bower components don't need to be in global-requirements:

- there are no other projects that use bower components, so we don't need
to ensure cross-project compatibility
- we can vet new versions of bower components as part of standard Horizon
change review


Richard


On 19 November 2014 06:43, Richard Jones r1chardj0...@gmail.com wrote:

 On 19 November 2014 03:14, Radomir Dopieralski openst...@sheep.art.pl
 wrote:

 On 18/11/14 12:54, Matthias Runge wrote:
  On Tue, Nov 18, 2014 at 09:36:00PM +1100, Richard Jones wrote:
  I guess I got the message that turning bower packages into system
 packages
  was something that the Linux packagers were not keen on. Did I get the
  message wrong there? If so, *and* we can get the bower stuff through
 #infra
  and global-requirements then yes, we should totally try to avoid
 adding the
  xstatic layer :)
 
  For me, it's more work to turn packages into bower, or to translate
  bower packages to system packages.
 
  But that is purely because we don't have bower packaged currently in
 Fedora.
  I would vote for the cleaner way (whatever that is).
 
  XStatic is obviously not the cleanest way, but a good compromise in most
  situations.

 The way I thought about it, we would simply replace the Bower packages
 with the corresponding system packages, by just changing the path in the
 settings.py file.

 Maybe an example would help illustrate it.
 Currently we have something like this:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular).base_dir),
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular_cookies).base_dir),
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular_mock).base_dir),
 ('horizon/lib/bootstrap_datepicker',
xstatic.main.XStatic(xstatic.pkg.bootstrap_datepicker).base_dir),
 ('bootstrap',
xstatic.main.XStatic(xstatic.pkg.bootstrap_scss).base_dir),
 ...
 ]

 We would replace that with:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
os.path.join(BASE_DIR, 'bower_modules/angular'),
 ...
 ]

 or wherever bower puts the files. Now, the packagers would replace those
 lines with:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
'/usr/lib/javascript/angular'),
 ...
 ]

 or whatever the packages in their distribution use. It's less work than
 having to make a whole xstatic package just to put that single path in
 there. The horizon system package would already depend on all the
 javascript library packages, so we are sure the files are there.
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 +1 to all that, except I'd recommend using django-bower to handle the
 static collection stuff. It's not documented but django-bower has a setting
 BOWER_COMPONENTS_ROOT which would make the above transition much simpler.
 You leave it alone for local dev, and packagers setup BOWER_COMPONENTS_ROOT
 to '/usr/lib/javascript/' or wherever.

 bower also has a concept of entry points - there a main value in the
 bower.json which identifies the Javascript, CSS, font and other files to
 include in the index.html to have the library loaded into your application.
 Saves a bunch of manual editing to get things right when you include the
 library. Sadly, the current django-bower plugin doesn't use any of that,
 though it does handle the collectstatic stuff.


  Richard


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On Fri Nov 21 2014 at 4:06:51 AM Thomas Goirand z...@debian.org wrote:

 On 11/17/2014 06:54 PM, Radomir Dopieralski wrote:
  - A tool, probably a script, that would help packaging the Bower
  packages into DEB/RPM packages. I suspect the Debian/Fedora packagers
  already have a semi-automatic solution for that.

 Nop. Bower isn't even packaged in Debian. Though I may try to do it
 (when I'm done with other Mirantis stuff like packaging Fuel for
 Debian...).


Just to be clear, it's not bower itself (the command-line tool) that needs
packaging, just the components that bower itself packages.



 On 11/18/2014 07:59 AM, Richard Jones wrote:
  I was envisaging us creating a tool which generates xstatic packages
  from bower packages. I'm not the first to think along these
  lines
 http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

 I think that's a very good idea!

  I wrote the tool today, and you can find it here:
 
  https://github.com/r1chardj0n3s/flaming-shame

 AWESOME ! :)
 Then now, everyone is happy. Thank you.


Well, no, but at least it exists ;)



 On 11/18/2014 04:22 PM, Radomir Dopieralski wrote:
  If we use Bower, we don't need to use Xstatic. It would be pure
  overhead. Bower already takes care of tracking releases and versions,
  and of bundling the files. All we need is a simple line in the
  settings.py telling Django where it puts all the files -- we don't
  really need Xstatic just for that. The packagers can then take those
  Bower packages and turn them into system packages, and just add/change
  the paths in settings.py to where they put the files. All in one
  place.

 The issue is that there's often not just a single path, but a full
 directory structure to address. That is easily managed with a Debian
 xstatic package, not sure how it would be with Bower.


I'm not sure what the difference is (unless it's just related to the
Debian-specific historical issue you raise below). xstatic and bower are
remarkably similar a directory to be packaged and some meta-data describing
it.



 On 11/18/2014 06:36 PM, Richard Jones wrote:
  I guess I got the message that turning bower packages into system
  packages was something that the Linux packagers were not keen on.

 What I'm not a fan of, is that we'll have external dependencies being
 bumped all the time, with unexpected consequences. At least, with
 xstatic packages, we control what's going on (though I understand the
 overhead work problem).


The dependencies won't be bumped any faster than reviewers allow, though I
realise that's not necessarily going to make you sleep any easier. Hmm.



 By the way, I went into bower.io as I wanted to have a look. How do I
 download a binary package for let's say jasmin? When searching, it
 just links to github...


Again; bower is not npm! Jasmine is a command-line program which is
packaged by npm. Bower packages bundles of stuff that are included in web
applications. bower itself, a command-line tool, is packaged by npm, but
itself manages other packages which are not command-line things, but just
bundles of css, javascript, images, fonts, etc. that are resources for web
applications to use.



 On 11/19/2014 12:14 AM, Radomir Dopieralski wrote:
  We would replace that with:
 
  STATICFILES_DIRS = [
  ('horizon/lib/angular',
 os.path.join(BASE_DIR, 'bower_modules/angular'),
  ...
  ]

 This would only work if upstream package directory structure is the same
 as the one in the distribution. For historical reason, that's
 unfortunately often not the case (sometimes because we want to keep
 backward compatibility in the distro because of reverse dependencies),
 and just changing the path wont make it work.

 On 11/19/2014 03:43 AM, Richard Jones wrote:
  +1 to all that, except I'd recommend using django-bower to handle the
  static collection stuff. It's not documented but django-bower has a
  setting BOWER_COMPONENTS_ROOT which would make the above transition much
  simpler. You leave it alone for local dev, and packagers setup
  BOWER_COMPONENTS_ROOT to '/usr/lib/javascript/' or wherever.

 s/lib/share/

 However, I'm almost sure that wont be enough to make it work. For
 example, in Debian, we have /usr/share/javascript/angular.js, not just
 /usr/share/javascript/angular. So django-bower would be searching on the
 wrong path.


That is unfortunate. It may be that Debian therefore has to special-case
angular to handle that case.

I think the general idea of following the component pattern set by bower
(separate directories with no risk of conflicts, and using the bower.json
meta-data to allow automatic configuration of the component) is too good to
dismiss though. Far less work for everyone, including packagers.

Perhaps the new packages should have bower in their name?



 On 11/19/2014 12:25 PM, Richard Jones wrote:
  In their view, bower components don't need to be in global-requirements:
 
  - there are no other projects that use bower components, so we

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

On 21 November 2014 16:12, Thomas Goirand z...@debian.org wrote:

 On 11/21/2014 10:52 AM, Richard Jones wrote:
  On 11/18/2014 04:22 PM, Radomir Dopieralski wrote:
   If we use Bower, we don't need to use Xstatic. It would be pure
   overhead. Bower already takes care of tracking releases and
 versions,
   and of bundling the files. All we need is a simple line in the
   settings.py telling Django where it puts all the files -- we don't
   really need Xstatic just for that. The packagers can then take
 those
   Bower packages and turn them into system packages, and just
 add/change
   the paths in settings.py to where they put the files. All in one
   place.
 
  The issue is that there's often not just a single path, but a full
  directory structure to address. That is easily managed with a Debian
  xstatic package, not sure how it would be with Bower.
 
 
  I'm not sure what the difference is (unless it's just related to the
  Debian-specific historical issue you raise below). xstatic and bower are
  remarkably similar a directory to be packaged and some meta-data
  describing it.

 Let me explain again then.

 Let's say there's python-xstatic-foo, and libjs-foo in Debian. If the
 directory structure of libjs-foo is very different from xstatic-foo, I
 can address that issue with symlinks within the xstatic package. Just
 changing the BASE_DIR may not be enough, as libjs-foo may have files
 organized in a very different way than in the upstream package for foo.


OK, so python-xstatic-foo can depend on libjs-foo and just symlink, fair
enough. I'm still not sure what makes bower unique in this respect,
although it'd be nice to avoid creating additional packages just to symlink
things around for bower, which I think is what you're getting at.


 By the way, I went into bower.io http://bower.io as I wanted to
  have a look. How do I
  download a binary package for let's say jasmin? When searching, it
  just links to github...
 
 
  Again; bower is not npm! Jasmine is a command-line program which is
  packaged by npm. Bower packages bundles of stuff that are included in
  web applications. bower itself, a command-line tool, is packaged by npm,
  but itself manages other packages which are not command-line things, but
  just bundles of css, javascript, images, fonts, etc. that are resources
  for web applications to use.

 Sure. But how do I download a bower package then?


I'm not sure I understand the meaning behind this question. bower install
angular downloads a bower package called angular.


 This would only work if upstream package directory structure is the
 same
  as the one in the distribution. For historical reason, that's
  unfortunately often not the case (sometimes because we want to keep
  backward compatibility in the distro because of reverse
 dependencies),
  and just changing the path wont make it work.
 
  On 11/19/2014 03:43 AM, Richard Jones wrote:
   +1 to all that, except I'd recommend using django-bower to handle
 the
   static collection stuff. It's not documented but django-bower has a
   setting BOWER_COMPONENTS_ROOT which would make the above
  transition much
   simpler. You leave it alone for local dev, and packagers setup
   BOWER_COMPONENTS_ROOT to '/usr/lib/javascript/' or wherever.
 
  s/lib/share/
 
  However, I'm almost sure that wont be enough to make it work. For
  example, in Debian, we have /usr/share/javascript/angular.__js, not
 just
  /usr/share/javascript/angular. So django-bower would be searching on
 the
  wrong path.
 
 
  That is unfortunate. It may be that Debian therefore has to special-case
  angular to handle that case.

 I wasn't making a point about Angular here. It's a general issue we have
 to take care of addressing correctly.

  I think the general idea of following the component pattern set by bower
  (separate directories with no risk of conflicts, and using the
  bower.json meta-data to allow automatic configuration of the component)
  is too good to dismiss though. Far less work for everyone, including
  packagers.
 
  Perhaps the new packages should have bower in their name?

 There's already libjs-angularjs and a bunch of python-xstatic-angular
 packages in Debian. I'm not sure that it is necessary to *also* have a
 bower-angularjs packages. Why would I need to do that? Just for the
 .json file? If that's the case, then couldn't I just add the bower.json
 file in the existing libjs-* packages? I'm not sure I get the point here...


The angular component that bower installs looks like this:

$ ls -CAF bower_components/angular
.bower.json angular-csp.css angular.min.js angular.min.js.map package.json
README.md angular.js angular.min.js.gzip bower.json

The bootstrap component looks like this:

$ ls -CAF bower_components/boot/
.bower.json LICENSE bower.json fonts/ js/ package.json
Gruntfile.js README.md dist/ grunt

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Richard Jones]

All the bower components I've used have included full source alongside any
minified versions.

On Sat, 22 Nov 2014 07:40 Fox, Kevin M kevin@pnnl.gov wrote:

 Simply having a git repository does not imply that its source.

 In fact, if its considered compiled (minified), I'm thinking the debian
 rules would prevent sourcing from it?

 Thanks,
 Kevin
 
 From: Donald Stufft [don...@stufft.io]
 Sent: Friday, November 21, 2014 8:39 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Horizon] the future of angularjs
 development  in Horizon

  On Nov 21, 2014, at 11:32 AM, Jeremy Stanley fu...@yuggoth.org wrote:
 
  On 2014-11-21 07:31:36 -0500 (-0500), Donald Stufft wrote:
  You can’t. Bower doesn’t have “traditional” packages where you take a
  directory and archive it using tar/zip/whatever and then upload it to
  some repo. Bower has a registry which maps names to git URLs and then
  the bower CLI looks up that mapping, fetches the git repository and then
  uses that as the input to the “look at metadata and do stuff with files”
  part of the package manager instead of the output of an un-unarchival
  command.
 
  This raises interesting free software philosophy/license
  questions... how do I redistribute (or even examine) the source of
  a bower-managed package? Is there a way without actually
  reverse-engineering the toolchain?

 Well it’s a git repository, so you could just clone it and look at it.

 ---
 Donald Stufft
 PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

Thanks everyone, I've closed the poll. I'm sorry to say that there's no
combination of two timeslots which allows everyone to attend a meeting. Of
the 25 respondents, the best we can do is cater for 24 of you.

Optimising for the maximum number of attendees, the potential meeting times
are 2000 UTC Tuesday and 1000 UTC on one of Monday, Wednesday or Friday. In
all three cases the only person who has indicated they cannot attend is
Lifeless.

Unfortunately, David has indicated that he can't be present at the Tuesday
2000 UTC slot. Optimising for him as a required attendee for both meetings
means we lose an additional attendee, and gives us the Wednesday 2000 UTC
slot and a few options:

- Monday, Wednesday and Thursday at 1200 UTC (Lifeless and ygbo miss)
- Friday at 1200 UTC (Lifeless and Jaromir Coufal miss)

If anyone else would like to play with the timeslots and numbers, I can
pass on the excel sheet and my code :)

According to the meetings wiki page, we should be able to get an IRC room
at any of the above times.


  Richard

On Wed Nov 19 2014 at 9:15:40 AM Richard Jones r1chardj0...@gmail.com
wrote:

 Thanks everyone who has responded. I'm going to leave the poll open until
 the weekend to allow for stragglers to get their times in, and then close
 it and we can see what the results are.

 The poll is at https://doodle.com/47h3f35nad62ncnf scroll to the far
 right to set your timezone.


  Richard
 On 12 November 2014 12:45, Richard Jones r1chardj0...@gmail.com wrote:

 I have set up a doodle poll to let folk enter their preferred times. It's
 in UTC/GMT (/London time, because doodle) so use something like
 http://everytimezone.com/ to figure that out :)

 https://doodle.com/47h3f35nad62ncnf


  Richard

 On 11 November 2014 18:46, Matthias Runge mru...@redhat.com wrote:

 On 11/11/14 08:09, Richard Jones wrote:
  Hi all,
 
  At the summit meetup last week I proposed that the Horizon weekly
  meeting time alternate between the current time and something more
  suitable for those of us closer to UTC+10. I'd like to get an
 indication
  of the interest in this, and I'll look into getting a second meeting
  time booked for alternating weeks based on your feedback.
 
  As a starting point, I'd like to suggest the times alternate between
 UTC
   and 1600 (the current time).

 Sadly, both times don't work for me. I would propose something like 8
 UTC, which should work for most folks located in EU and east, or 18 UTC.

 Matthias


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] proposal: alternating weekly meeting time [doodle poll created]

[Richard Jones]

Thanks!

On Tue Nov 25 2014 at 3:15:49 AM Yves-Gwenaël Bourhis 
yves-gwenael.bour...@cloudwatt.com wrote:



 Le 24/11/2014 04:20, Richard Jones a écrit :
  Thanks everyone, I've closed the poll. I'm sorry to say that there's no
  combination of two timeslots which allows everyone to attend a meeting.
  Of the 25 respondents, the best we can do is cater for 24 of you.
 
  Optimising for the maximum number of attendees, the potential meeting
  times are 2000 UTC Tuesday and 1000 UTC on one of Monday, Wednesday or
  Friday. In all three cases the only person who has indicated they cannot
  attend is Lifeless.
 
  Unfortunately, David has indicated that he can't be present at the
  Tuesday 2000 UTC slot. Optimising for him as a required attendee for
  both meetings means we lose an additional attendee, and gives us the
  Wednesday 2000 UTC slot and a few options:
 
  - Monday, Wednesday and Thursday at 1200 UTC (Lifeless and ygbo miss)

 1200 UTC is perfect for me.
 The doodle was proposing 1200 UTC to 1400 UTC and in the 2 hours
 bandwidth I can not be sure to be there. but if it's 1200 on the spot
 I can for sure :-)
 Since I couldn't precise this on the doodle I didn't select this slot. A
 one hour bandwidth would have allowed more precision but I understand
 you concern that the doodle would have been to long to scroll.

 --
 Yves-Gwenaël Bourhis

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] REST and Django

[Richard Jones]

On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S travis.tr...@hp.com
wrote:

  Hi Richard,

  You are right, we should put this out on the main ML, so copying thread
 out to there.  ML: FYI that this started after some impromptu IRC
 discussions about a specific patch led into an impromptu google hangout
 discussion with all the people on the thread below.


Thanks Travis!



 As I mentioned in the review[1], Thai and I were mainly discussing the
 possible performance implications of network hops from client to horizon
 server and whether or not any aggregation should occur server side.   In
 other words, some views  require several APIs to be queried before any data
 can displayed and it would eliminate some extra network requests from
 client to server if some of the data was first collected on the server side
 across service APIs.  For example, the launch instance wizard will need to
 collect data from quite a few APIs before even the first step is displayed
 (I’ve listed those out in the blueprint [2]).

  The flip side to that (as you also pointed out) is that if we keep the
 API’s fine grained then the wizard will be able to optimize in one place
 the calls for data as it is needed. For example, the first step may only
 need half of the API calls. It also could lead to perceived performance
 increases just due to the wizard making a call for different data
 independently and displaying it as soon as it can.


Indeed, looking at the current launch wizard code it seems like you
wouldn't need to load all that data for the wizard to be displayed, since
only some subset of it would be necessary to display any given panel of the
wizard.



 I tend to lean towards your POV and starting with discrete API calls and
 letting the client optimize calls.  If there are performance problems or
 other reasons then doing data aggregation on the server side could be
 considered at that point.


I'm glad to hear it. I'm a fan of optimising when necessary, and not
beforehand :)



 Of course if anybody is able to do some performance testing between the
 two approaches then that could affect the direction taken.


I would certainly like to see us take some measurements when performance
issues pop up. Optimising without solid metrics is bad idea :)


Richard



  [1]
 https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py
 [2]
 https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign

  -Travis

   From: Richard Jones r1chardj0...@gmail.com
 Date: Wednesday, November 26, 2014 at 11:55 PM
 To: Travis Tripp travis.tr...@hp.com, Thai Q Tran/Silicon Valley/IBM 
 tqt...@us.ibm.com, David Lyle dkly...@gmail.com, Maxime Vidori 
 maxime.vid...@enovance.com, Wroblewski, Szymon 
 szymon.wroblew...@intel.com, Wood, Matthew David (HP Cloud - Horizon) 
 matt.w...@hp.com, Chen, Shaoquan sean.ch...@hp.com, Farina, Matt
 (HP Cloud) matthew.far...@hp.com, Cindy Lu/Silicon Valley/IBM 
 c...@us.ibm.com, Justin Pomeroy/Rochester/IBM jpom...@us.ibm.com, Neill
 Cox neill@ingenious.com.au
 Subject: Re: REST and Django

  I'm not sure whether this is the appropriate place to discuss this, or
 whether I should be posting to the list under [Horizon] but I think we need
 to have a clear idea of what goes in the REST API and what goes in the
 client (angular) code.

  In my mind, the thinner the REST API the better. Indeed if we can get
 away with proxying requests through without touching any *client code, that
 would be great.

  Coding additional logic into the REST API means that a developer would
 need to look in two places, instead of one, to determine what was happening
 for a particular call. If we keep it thin then the API presented to the
 client developer is very, very similar to the API presented by the
 services. Minimum surprise.

  Your thoughts?


   Richard


 On Wed Nov 26 2014 at 2:40:52 PM Richard Jones r1chardj0...@gmail.com
 wrote:

 Thanks for the great summary, Travis.

  I've completed the work I pledged this morning, so now the REST API
 change set has:

  - no rest framework dependency
 - AJAX scaffolding in openstack_dashboard.api.rest.utils
 - code in openstack_dashboard/api/rest/
 - renamed the API from identity to keystone to be consistent
 - added a sample of testing, mostly for my own sanity to check things
 were working

  https://review.openstack.org/#/c/136676


Richard

 On Wed Nov 26 2014 at 12:18:25 PM Tripp, Travis S travis.tr...@hp.com
 wrote:

  Hello all,

  Great discussion on the REST urls today! I think that we are on track
 to come to a common REST API usage pattern.  To provide quick summary:

  We all agreed that going to a straight REST pattern rather than
 through tables was a good idea. We discussed using direct get / post in
 Django views like what Max originally used[1][2] and Thai also started[3]
 with the identity table rework or to go with djangorestframework [5] like
 what Richard was prototyping with[4].

  The main things we would

Re: [openstack-dev] [horizon] REST and Django

[Richard Jones]

On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran tqt...@us.ibm.com wrote:

 I agree that keeping the API layer thin would be ideal. I should add that
 having discrete API calls would allow dynamic population of table. However,
 I will make a case where it *might* be necessary to add additional APIs.
 Consider that you want to delete 3 items in a given table.

 If you do this on the client side, you would need to perform: n * (1 API
 request + 1 AJAX request)
 If you have some logic on the server side that batch delete actions: n *
 (1 API request) + 1 AJAX request

 Consider the following:
 n = 1, client = 2 trips, server = 2 trips
 n = 3, client = 6 trips, server = 4 trips
 n = 10, client = 20 trips, server = 11 trips
 n = 100, client = 200 trips, server 101 trips

 As you can see, this does not scale very well something to consider...

Yep, though in the above cases the client is still going to be hanging,
waiting for those server-backend calls, with no feedback until it's all
done. I would hope that the client-server call overhead is minimal, but I
guess that's probably wishful thinking when in the land of random Internet
users hitting some provider's Horizon :)

So yeah, having mulled it over myself I agree that it's useful to have
batch operations implemented in the POST handler, the most common operation
being DELETE.

Maybe one day we could transition to a batch call with user feedback using
a websocket connection.


 Richard

 [image: Inactive hide details for Richard Jones ---11/27/2014 05:38:53
 PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S travis.tr]Richard
 Jones ---11/27/2014 05:38:53 PM---On Fri Nov 28 2014 at 5:58:00 AM Tripp,
 Travis S travis.tr...@hp.com wrote:

 From: Richard Jones r1chardj0...@gmail.com
 To: Tripp, Travis S travis.tr...@hp.com, OpenStack List 
 openstack-dev@lists.openstack.org
 Date: 11/27/2014 05:38 PM
 Subject: Re: [openstack-dev] [horizon] REST and Django
 --




 On Fri Nov 28 2014 at 5:58:00 AM Tripp, Travis S *travis.tr...@hp.com*
 travis.tr...@hp.com wrote:

Hi Richard,

You are right, we should put this out on the main ML, so copying
thread out to there.  ML: FYI that this started after some impromptu IRC
discussions about a specific patch led into an impromptu google hangout
discussion with all the people on the thread below.


 Thanks Travis!



As I mentioned in the review[1], Thai and I were mainly discussing the
possible performance implications of network hops from client to horizon
server and whether or not any aggregation should occur server side.   In
other words, some views  require several APIs to be queried before any data
can displayed and it would eliminate some extra network requests from
client to server if some of the data was first collected on the server side
across service APIs.  For example, the launch instance wizard will need to
collect data from quite a few APIs before even the first step is displayed
(I’ve listed those out in the blueprint [2]).

The flip side to that (as you also pointed out) is that if we keep the
API’s fine grained then the wizard will be able to optimize in one place
the calls for data as it is needed. For example, the first step may only
need half of the API calls. It also could lead to perceived performance
increases just due to the wizard making a call for different data
independently and displaying it as soon as it can.


 Indeed, looking at the current launch wizard code it seems like you
 wouldn't need to load all that data for the wizard to be displayed, since
 only some subset of it would be necessary to display any given panel of the
 wizard.



I tend to lean towards your POV and starting with discrete API calls
and letting the client optimize calls.  If there are performance problems
or other reasons then doing data aggregation on the server side could be
considered at that point.


 I'm glad to hear it. I'm a fan of optimising when necessary, and not
 beforehand :)



Of course if anybody is able to do some performance testing between
the two approaches then that could affect the direction taken.


 I would certainly like to see us take some measurements when performance
 issues pop up. Optimising without solid metrics is bad idea :)


 Richard




[1]

 *https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py*

 https://review.openstack.org/#/c/136676/8/openstack_dashboard/api/rest/urls.py
[2]
*https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign*
https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign

-Travis

*From: *Richard Jones *r1chardj0...@gmail.com*
r1chardj0...@gmail.com
 * Date: *Wednesday, November 26, 2014 at 11:55 PM
 * To: *Travis Tripp *travis.tr...@hp.com* travis.tr...@hp.com, Thai Q
Tran/Silicon Valley/IBM *tqt...@us.ibm.com* tqt...@us.ibm.com

[openstack-dev] [Horizon] REST API split and new requirements.txt

[Richard Jones]

Hi all, just to let you know that on request from Thai I split the REST API
change into two:

https://review.openstack.org/#/c/136676
  - original change which just has the base code

https://review.openstack.org/#/c/139532
  - keystone specifics

The identity WIP should now base itself off the second change.

I've also submitted a change (https://review.openstack.org/#/c/139284/)
with the new angular dependencies to go in requirements.txt, based on a
couple of xstatic packages I created today. Also created a change for
global-requirements as well. Eventually those will be superseded by bower,
but for now it's xstatic. If anyone would like to be added to PyPI to be
able to update those xstatic packages, please ask.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] REST and Django

[Richard Jones]

Sorry I didn't respond to this earlier today, I had intended to.

What you're describing isn't REST, and the principles of REST are what have
been guiding the design of the new API so far. I see a lot of value in
using REST approaches, mostly around clarity of the interface.

While the idea of a very thin proxy seemed like a great idea at one point,
my conversations at the summit convinced me that there was value in both
using the client interfaces present in the openstack_dashboard/api code
base (since they abstract away many issues in the apis including across
versions) and also value in us being able to clean up (for example, using
project_id rather than project in the user API we've already
implemented) and extend those interfaces (to allow batched operations).

We want to be careful about what we expose in Horizon to the JS clients
through this API. That necessitates some amount of code in Horizon. About
half of the current API for keysone represents that control (the other half
is docstrings :)


 Richard


On Tue Dec 09 2014 at 9:37:47 PM Tihomir Trifonov t.trifo...@gmail.com
wrote:

 Sorry for the late reply, just few thoughts on the matter.

 IMO the REST middleware should be as thin as possible. And I mean thin in
 terms of processing - it should not do pre/post processing of the requests,
 but just unpack/pack. So here is an example:

 instead of making AJAX calls that contain instructions:

 ​​
 POST --json --data {action: delete, data: [ {name:
 item1}, {name: item2}, {name: item3 ]}


 I think a better approach is just to pack/unpack batch commands, and leave
 execution to the frontend/backend and not middleware:


 ​​
 POST --json --data {
 ​batch
 :
 ​[
 {​
 
 ​
 action
 ​ : delete​
 ,
 ​payload: ​
 {name: item1}
 ​,
 {​
 
 ​
 action
 ​ : delete​
 ,
 ​
 payload:
 ​
 {name: item
 ​2
 }
 ​,
 {​
 
 ​
 action
 ​ : delete​
 ,
 ​
 payload:
 ​
 {name: item
 ​3
 }
 ​ ] ​
 ​
 ​
 }


 ​The idea is that the middleware should not know the actual data. It
 should ideally just unpack the data:

 ​​responses = []


 for cmd in
 ​ ​
 ​
 ​
 request.POST['batch']:​


 ​
 ​​responses
 ​.append(​
 ​
 getattr(controller, cmd['action']
 ​)(**
 cmd['​payload']
 ​)​)


 ​return responses​



 ​and the frontend(JS) will just send batches of simple commands, and will
 receive a list of responses for each command in the batch. The error
 handling will be done in the frontend​(JS) as well.

 ​

 For the more complex example of 'put()' where we have dependent objects:

 project = api.keystone.tenant_get(request, id)
 kwargs = self._tenant_kwargs_from_DATA(request.DATA, enabled=None)
 api.keystone.tenant_update(request, project, **kwargs)



 In practice the project data should be already present in the
 frontend(assuming that we already loaded it to render the project
 form/view), so

 ​
 ​
 POST --json --data {
 ​batch
 :
 ​[
 {​
 
 ​
 action
 ​ : tenant_update​
 ,
 ​payload: ​
 {project: js_project_object.id, name: some name, prop1: some
 prop, prop2: other prop, etc.}
 ​
 ​ ] ​
 ​
 ​
 }​

 So in general we don't need to recreate the full state on each REST call,
 if we make the Frontent full-featured application. This way - the frontend
 will construct the object, will hold the cached value, and will just send
 the needed requests as single ones or in batches, will receive the response
 from the API backend, and will render the results. The whole processing
 logic will be held in the Frontend(JS), while the middleware will just act
 as proxy(un/packer). This way we will maintain just the logic in the
 frontend, and will not need to duplicate some logic in the middleware.




 On Tue, Dec 2, 2014 at 4:45 PM, Adam Young ayo...@redhat.com wrote:

  On 12/02/2014 12:39 AM, Richard Jones wrote:

 On Mon Dec 01 2014 at 4:18:42 PM Thai Q Tran tqt...@us.ibm.com wrote:

  I agree that keeping the API layer thin would be ideal. I should add
 that having discrete API calls would allow dynamic population of table.
 However, I will make a case where it *might* be necessary to add
 additional APIs. Consider that you want to delete 3 items in a given table.

 If you do this on the client side, you would need to perform: n * (1 API
 request + 1 AJAX request)
 If you have some logic on the server side that batch delete actions: n *
 (1 API request) + 1 AJAX request

 Consider the following:
 n = 1, client = 2 trips, server = 2 trips
 n = 3, client = 6 trips, server = 4 trips
 n = 10, client = 20 trips, server = 11 trips
 n = 100, client = 200 trips, server 101 trips

 As you can see, this does not scale very well something to
 consider...

  This is not something Horizon can fix.  Horizon can make matters worse,
 but cannot make things better.

 If you want to delete 3 users,   Horizon still needs to make 3 distinct
 calls to Keystone.

 To fix this, we need either batch calls or a standard way to do multiples
 of the same operation.

 The unified API effort it the right place to drive this.







  Yep, though

Re: [openstack-dev] Moving _conf and _scripts to dashboard

[Richard Jones]

+1 to moving application configuration to the application, out of the
library.


 Richard

On Thu Dec 11 2014 at 10:38:20 AM Thai Q Tran tqt...@us.ibm.com wrote:

 The way we are structuring our javascripts today is complicated. All of
 our static javascripts reside in /horizon/static and are imported through
 _conf.html and _scripts.html. Notice that there are already some panel
 specific javascripts like: horizon.images.js, horizon.instances.js,
 horizon.users.js. They do not belong in horizon. They belong in
 openstack_dashboard because they are specific to a panel.

 Why am I raising this issue now? In Angular, we need controllers written
 in javascript for each panel. As we angularize more and more panels, we
 need to store them in a way that make sense. To me, it make sense for us to
 move _conf and _scripts to openstack_dashboard. Or if this is not possible,
 then provide a mechanism to override them in openstack_dashboard.

 Thoughts?
 Thai


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] REST and Django

[Richard Jones]

On Fri Dec 12 2014 at 1:06:08 PM Tripp, Travis S travis.tr...@hp.com
wrote:

 ​Do we really need the lines:​

  project = api.keystone.tenant_get(request, id)
  kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
 ​
 I agree that if you already have all the data it is really bad to have to
 do another call. I do think there is room for discussing the reasoning,
 though.
 As far as I can tell, they do this so that if you are updating an entity,
 you have to be very specific about the fields you are changing. I actually
 see this as potentially a protectionary measure against data loss and a
 sometimes very nice to have feature. It perhaps was intended to *help*
 guard against race conditions *sometimes*.


Yep, it looks like I broke this API by implementing it the way I did, and
I'll alter the API so that you pass both the current object (according to
the client) and the parameters to alter.

Thanks everyone for the great reviewing!


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

This is a good proposal, though I'm unclear on how the static_settings.py
file is populated by a developer (as opposed to a packager, which you
described).


 Richard

On Fri Dec 19 2014 at 12:59:37 AM Radomir Dopieralski 
openst...@sheep.art.pl wrote:

 Hello,

 revisiting the package management for the Horizon's static files again,
 I would like to propose a particular solution. Hopefully it will allow
 us to both simplify the whole setup, and use the popular tools for the
 job, without losing too much of benefits of our current process.

 The changes we would need to make are as follows:

 * get rid of XStatic entirely;
 * add to the repository a configuration file for Bower, with all the
 required bower packages listed and their versions specified;
 * add to the repository a static_settings.py file, with a single
 variable defined, STATICFILES_DIRS. That variable would be initialized
 to a list of pairs mapping filesystem directories to URLs within the
 /static tree. By default it would only have a single mapping, pointing
 to where Bower installs all the stuff by default;
 * add a line from static_settings import STATICFILES_DIRS to the
 settings.py file;
 * add jobs both to run_tests.sh and any gate scripts, that would run Bower;
 * add a check on the gate that makes sure that all direct and indirect
 dependencies of all required Bower packages are listed in its
 configuration files (pretty much what we have for requirements.txt now);

 That's all. Now, how that would be used.

 1. The developers will just use Bower the way they would normally use
 it, being able to install and test any of the libraries in any versions
 they like. The only additional thing is that they would need to add any
 additional libraries or changed versions to the Bower configuration file
 before they push their patch for review and merge.

 2. The packagers can read the list of all required packages from the
 Bower configuration file, and make sure they have all the required
 libraries packages in the required versions.

 Next, they replace the static_settings.py file with one they have
 prepared manually or automatically. The file lists the locations of all
 the library directories, and, in the case when the directory structure
 differs from what Bower provides, even mappings between subdirectories
 and individual files.

 3. Security patches need to go into the Bower packages directly, which
 is good for the whole community.

 4. If we aver need a library that is not packaged for Bower, we will
 package it just as we had with the XStatic packages, only for Bower,
 which has much larger user base and more chance of other projects also
 using that package and helping with its testing.

 What do you think? Do you see any disastrous problems with this system?
 --
 Radomir Dopieralski

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

On Wed Jan 21 2015 at 7:00:12 AM Matthew Farina m...@mattfarina.com wrote:

 Radomir, maybe you can help me better understand where this would go. I
 have a few questions.

 First, can you point me to a time when horizon used system packages
 successfully for JavaScript libraries? When I looked through the Debian and
 Ubuntu packages I couldn't find the libraries horizon is using. I'm curious
 to see this in action.


I'm not a user of these packaging systems, but I found this, which I
believe shows the javascript system packages used:

https://packages.debian.org/sid/openstack-dashboard



 Front-end systems almost never use system packagers like this. Can you
 point me to applications like horizon that use system packages this way? If
 Horizon is going to go it virtually alone in this space, what will that
 mean for our level of work and ability to have updates?


Horizon is not a typical web front end system, it's part of OpenStack, and
thus has to abide by OpenStack packaging conventions. Those dictate that we
must have Horizon packaged by system packagers for deployment.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

On Fri Jan 23 2015 at 4:28:59 AM Matthew Farina m...@mattfarina.com wrote:

 I would like to add one more nuance to this discussion that I don't
 remember seeing.

 JavaScript libraries run in web browser in their JavaScript engines (like
 v8) rather than on the server. A version of a JS library may be fine on a
 system, without any security issues, but contain browser issues. The
 version used matters more to the application and the web browsers consuming
 the application than to the system it's on.


I think you're confusing Javascript in the browser vs. Javascript for
node.js

Those are two separate things. Javascript used in the browser is rarely
(never in our case) used also for node.js development. There are even two
separate packaging systems: bower is all about components for browsers,
whereas npm is all about packages for node.js. Our discussion is about
bower, and definitely not about npm or the node.js programming language.



 Some of the libraries exist as packages. For example, there are some
 debian packages. These have older versions of libraries than those that
 will work in Horizon.


Thanks to packagers working with OpenStack we are able to get newer
versions of the components we need once we have pinned them in our code.



 The libraries need to integrate for horizon and the browsers. So,
 supporting varying versions of a js library, their interactions together,
 and creating a usable interface will be a real problem. For example the
 debian packages of old or varying versions will a problem for those of us
 attempting to craft a UI. What's there isn't practically usable today. Some
 things are missing.


It's entirely up to us to specify a pinned set of component versions that
we need, which we then need the system packagers to ensure are available as
deb or rpm. Gaps will be filled as part of the usual packaging efforts that
go on during the OpenStack release cycle.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

On Mon Jan 05 2015 at 7:59:14 PM Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 05/01/15 00:35, Richard Jones wrote:
  On Mon Dec 22 2014 at 8:24:03 PM Radomir Dopieralski
  openst...@sheep.art.pl mailto:openst...@sheep.art.pl wrote:
 
  On 20/12/14 21:25, Richard Jones wrote:
   This is a good proposal, though I'm unclear on how the
   static_settings.py file is populated by a developer (as opposed to
 a
   packager, which you described).
 
  It's not, the developer version is included in the repository, and
  simply points to where Bower is configured to put the files.
  So just to be clear, as developers we:
 
  1. have a bower.json listing the bower component to use,
  2. use bower to fetch and install those to the bower_components
  directory at the top level of the Horizon repos checkout, and
  3. manually edit static_settings.py when we add a new bower component to
  bower.json so it knows the appropriate static files to load from that
  component.
 
  Is that correct?
 
  The above will increase the burden on those adding or upgrading bower
  components (they'll need to check the bower.json in the component for
  the appropriate static files to link in) but will make life easier for
  the re-packagers since they'll know which files they need to cater for
  in static_settings.py

 Well, I expect you can tell Bower to put the files somewhere else than
 in the root directory of the project -- a directory like ``bower_files``
 or something (that directory is also added to ``.gitignore`` so that you
 don't commit it by mistake). Then only that directory needs to be added
 to the ``static_settings.py``. Of course, you still need to make all the
 ``script`` links in appropriate places with the right URLs, but you
 would have to do that anyways.


Bower installs into a directory called bower_components in the current
directory, which is equivalent to your bower_files above.



 Let's look at an example. Suppose you need to a new JavaScript library
 called hipster.js. You add it to the ``bower.json`` file, and run
 Bower. Bower downloads the right files and does whatever it is that it
 does to them, and puts them in  ``bower_files/hipster-js``. Now you edit
 Horizon's templates and add ``script src={{ STATIC_URL
 }}/hipster-js/hipster.js`` to ``_scripts.html``. That's it for you.
 Since your ``static_settings.py`` file already has a line:

   ('', os.path.join(BASE_DIR, '/bower_files')),

 in it, it will just work.


Yep, except s/bower_files/bower_components :)


Now, suppose that a packager wants to package this for, say, Debian. And
 suppose that Debian has hipster.js packaged, except it was called
 bro.js before, and they left the old name for compatibility reasons.
 He will look at the change history to the ``bower.json`` and the
 ``_scripts.html`` files, take the ``static_settings.py`` file for his
 distribution, and add a line:

   ('hipster-js/hipster.js', '/usr/lib/js_libraries/bro_js/bro.js')


Ah! I had forgotten about that feature. Yep, all good :)


  Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

I think the only outstanding question is how developers and non-packagers
populate the bower_components directory - that is, how is bower expected to
be available for them?

I think following the Storyboard approach is a good idea: isolate a
known-working node/bower environment local to horizon which is managed by
tox - so to invoke bower you run tox -e bower command. No worries about
system installation or compatibility, and works in the gate.

Horizon installation (whenever a pip install would be invoked) would then
also have a tox -e bower install invocation.

Storyboard[1] uses a thing called nodeenv[2] which is installed through pip
/ requirements.txt to control the node environment. It then has bower
commands in tox.ini[3] (though I'd just have a single bower environment
to implement the tox command I suggest above.


 Richard

[1] https://wiki.openstack.org/wiki/StoryBoard
[2] https://pypi.python.org/pypi/nodeenv
[3]
https://git.openstack.org/cgit/openstack-infra/storyboard-webclient/tree/tox.ini


On Tue Jan 06 2015 at 11:42:17 AM Tripp, Travis S travis.tr...@hp.com
wrote:

 What Radomir proposes looks like it would greatly ease the process I am
 still going through to get the latest angular available to Horizon for
 current development.  At the time of writing this, I’m still trying to get
 the updated library through.  I hit a rather difficult workflow:


   1.  Packaged the latest into Xstatic-Angular-1.3.7
   2.  Submitted patch which deprecated the separate older
 xstatic-angular-cookies and xstatic-angular-mock packages
   3.  Reviewed and approved (after correcting an initial mis-repackaging)
   4.  Radomir released to Pypi

 This was pretty easy and not too hard. Not too much to complain about.

 However, now, to get Horizon to use it, I have to get that into global
 requirements.  Since I’m deprecating old packages I got stuck in a sort of
 ugly dependency path.  I couldn’t remove the cookies and mock libraries
 from the global requirements patch that added the new 1.3.7 package because
 of horizon still referencing the deprecated packages.  And, when I did it
 anyway, the integration tests failed due to horizon being dependent on
 something not in global requirements.  So, now, as far as I can tell we
 have to jump through the following hoops:


   1.  Global requirements patch to add angular 1.3.7
  *   Verify check / recheck fun
  *   Reviewed and approved
  *   Gate check / recheck fun
   2.  Horizon patch to update to angular 1.3.7 and remove deprecated mock
 and cookies packages
  *   Verify check / recheck fun
  *   Reviewed and approved
  *   Gate check / recheck fun
   3.  Global requirements patch to remove deprecated mock and cookies
  *   Verify check / recheck fun
  *   Reviewed and approved
  *   Gate check / recheck fun

 Don’t get me wrong, I really do think the gate is brilliant and am all for
 a review / approval process, but this does seem excessive for a UI library
 that should only be used by Horizon. Is there some other reason that this
 should have to go through global requirements?

 Thanks,
 Travis

 From: Richard Jones r1chardj0...@gmail.commailto:r1chardj0...@gmail.com
 
 Reply-To: OpenStack List openstack-dev@lists.openstack.orgmailto:
 openstack-dev@lists.openstack.org
 Date: Monday, January 5, 2015 at 2:08 AM
 To: OpenStack List openstack-dev@lists.openstack.orgmailto:openstack
 -d...@lists.openstack.org
 Subject: Re: [openstack-dev] [horizon] static files handling, bower/



 On Mon Jan 05 2015 at 7:59:14 PM Radomir Dopieralski 
 openst...@sheep.art.plmailto:openst...@sheep.art.pl wrote:
 On 05/01/15 00:35, Richard Jones wrote:
  On Mon Dec 22 2014 at 8:24:03 PM Radomir Dopieralski
  openst...@sheep.art.plmailto:openst...@sheep.art.pl mailto:
 openst...@sheep.art.plmailto:openst...@sheep.art.pl wrote:
 
  On 20/12/14 21:25, Richard Jones wrote:
   This is a good proposal, though I'm unclear on how the
   static_settings.py file is populated by a developer (as opposed to
 a
   packager, which you described).
 
  It's not, the developer version is included in the repository, and
  simply points to where Bower is configured to put the files.
  So just to be clear, as developers we:
 
  1. have a bower.json listing the bower component to use,
  2. use bower to fetch and install those to the bower_components
  directory at the top level of the Horizon repos checkout, and
  3. manually edit static_settings.py when we add a new bower component to
  bower.json so it knows the appropriate static files to load from that
  component.
 
  Is that correct?
 
  The above will increase the burden on those adding or upgrading bower
  components (they'll need to check the bower.json in the component for
  the appropriate static files to link in) but will make life easier for
  the re-packagers since they'll know which files they need to cater for
  in static_settings.py

 Well, I expect you can tell Bower to put

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

Thanks, Radomir. How much detail from this discussion should be captured in
the blueprint? I'm afraid I'm more familiar with the Python PEP process.

On Thu Jan 08 2015 at 11:38:57 PM Radomir Dopieralski 
openst...@sheep.art.pl wrote:

 On 06/01/15 01:53, Richard Jones wrote:
  I think the only outstanding question is how developers and
  non-packagers populate the bower_components directory - that is, how is
  bower expected to be available for them?
 
  I think following the Storyboard approach is a good idea: isolate a
  known-working node/bower environment local to horizon which is managed
  by tox - so to invoke bower you run tox -e bower command. No worries
  about system installation or compatibility, and works in the gate.
 
  Horizon installation (whenever a pip install would be invoked) would
  then also have a tox -e bower install invocation.
 
  Storyboard[1] uses a thing called nodeenv[2] which is installed through
  pip / requirements.txt to control the node environment. It then has
  bower commands in tox.ini[3] (though I'd just have a single bower
  environment to implement the tox command I suggest above.
 
 
   Richard
 
  [1] https://wiki.openstack.org/wiki/StoryBoard
  [2] https://pypi.python.org/pypi/nodeenv
  [3] https://git.openstack.org/cgit/openstack-infra/
 storyboard-webclient/tree/tox.ini
 

 I created a blueprint for this.
 https://blueprints.launchpad.net/horizon/+spec/static-file-bower
 --
 Radomir Dopieralski


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Richard Jones]

So just to be clear, as developers we:

1. have a bower.json listing the bower component to use,
2. use bower to fetch and install those to the bower_components directory
at the top level of the Horizon repos checkout, and
3. manually edit static_settings.py when we add a new bower component to
bower.json so it knows the appropriate static files to load from that
component.

Is that correct?

The above will increase the burden on those adding or upgrading bower
components (they'll need to check the bower.json in the component for the
appropriate static files to link in) but will make life easier for the
re-packagers since they'll know which files they need to cater for in
static_settings.py


  Richard


On Mon Dec 22 2014 at 8:24:03 PM Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 20/12/14 21:25, Richard Jones wrote:
  This is a good proposal, though I'm unclear on how the
  static_settings.py file is populated by a developer (as opposed to a
  packager, which you described).

 It's not, the developer version is included in the repository, and
 simply points to where Bower is configured to put the files.

 --
 Radomir Dopieralski


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Rethinking the launch-instance wizard model

[Richard Jones]

Hi folks,

Currently the launch instance model file does all the fetching of various
bits of data. Combined with all of the controllers also being loaded at
wizard startup, this results in some very difficult synchronisation issues*.

An issue I've run into is the initialisation of the controller based on
model data. Specifically, loading the allocated and available lists
into the security groups transfer table. I can load a reference to the
model securityGroups array as the available set, but since that data is
generally not loaded (by some other code) when the controller is setting
up, I can't also select the first group in the array as the default group
in allocated.

So, I propose that the model data for a particular pane be loaded *by that
pane*, so that pane can then attach a callback to run once the data is
loaded, to handle situations like this (which will be common, IIUC). Or the
model needs to provide promises for the pane controllers to attach
callbacks to.


  Richard

* one issue is the problem of the controllers running for the life of the
wizard and not knowing when they're active (having them only be temporarily
active would solve the issue of having to watch the transfer tables for
changes of data - we could just read the allocated lists when the
controller exits).
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] bower

[Richard Jones]

On Sun, 8 Mar 2015 at 04:59 Michael Krotscheck krotsch...@gmail.com wrote:

 Anyone wanna hack on a bower mirror puppet module with me?


BTW are you aware of this spec for bower/Horizon/infra?

https://review.openstack.org/#/c/154297/


Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon]Informal Pre-summit get together

[Richard Jones]

I'm planning on wearing my openstack badge, to make things a little easier
for those I don't spend all week on hangouts with ;)

On Sun, 17 May 2015 at 16:33 Michael Krotscheck krotsch...@gmail.com
wrote:

 Do you have any distinguishing features? Like an enormous openstack hat?

 On Sun, May 17, 2015, 5:06 AM Doug Fish the.doug.f...@gmail.com wrote:

 I missed the discussion in IRC. I'm going to be there Sunday. If we also
 go out Monday I'll probably show up then too

 Doug

  On May 17, 2015, at 12:40 AM, Tripp, Travis S travis.tr...@hp.com
 wrote:
 
  Just to double check, there was some IRC discussion about Monday.  So is
  it still Sunday?
 
  On 5/14/15, 3:41 PM, Douglas Fish drf...@us.ibm.com wrote:
 
 
  The Horizon team will be meeting on Sunday night informally over a beer
  and
  dinner:
  Sunday 6pm @  The Charles Bar http://thecharlesbar.ca/
 
  Hope to see you there!
 
  Doug
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] high priority patches, please review

[Richard Jones]

Hi Horizon devs,

The following test patches are a high priority, blocking further new work
in Liberty until they're landed. Please consider helping review them to get
the landed ASAP:

https://review.openstack.org/#/c/170554/
https://review.openstack.org/#/c/167738/
https://review.openstack.org/#/c/172057/
https://review.openstack.org/#/c/178227/
https://review.openstack.org/#/c/176532/
https://review.openstack.org/#/c/178434/
https://review.openstack.org/#/c/167326/


Cheers,

 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Angular dependencies and the hz.dashboard namespace

[Richard Jones]

The Adding Angular Identity Dashboard[1] patch exposed an issue I saw
previously that worried me[2].

I believe during recent Horizon work the concept of angular module
namespaces and dependencies have been conflated. There's this idea that if
you create a submodule inside a module namespace you *must* have that
module depend on that submodule. I believe that is incorrect - just look at
the angular codebase itself, and how it is used. If you want the ngCookies
module in a couple of places then you have those modules depend on
ngCookies (or, more likely, you just add it as a dependency to the app).
The point is it's not just added automatically to the ng module as a
dependency. If you need to use a module's functionality, you depend on the
module. You don't just have all your modules *automatically* pulled in as
dependencies. I have proposed a patch to remedy this for the existing
optional[3] project dashboard[4].

I believe it is unnecessary to add extension of the hz.dashboard dependency
list[5] since we already have an extensible dependency list for the
application itself (which the hz.dashboard dependency list just extends).

To answer the question explicitly raised what is the point of having the
hz.dashboard module if it has no dependencies? It does two things: it
defines the namespace in a formal manner (by itself unnecessary, but it's
still nice to do) and it defines a constant which is used by other code.
Eventually it may define more. There is an important difference between
Python modules and Angular modules here - using a Python module like
hz.dashboard in this way could cause problems because of the way Python
sub-module namespaces and import ordering work. Angular's modules work very
differently, and are not burdened by the same issues. In Python that
constant would most likely have to be pushed out to a sub-module to avoid
import loops.


 Richard

[1] https://review.openstack.org/190852
[2] https://bugs.launchpad.net/horizon/+bug/1466713
[3] There may be other issues that prevent the project dashboard being
optional - there are dependencies in Python-land from the admin dashboard
hard-coded over to the project dashboard, for example. It might be a good
idea to remedy that, since I think it probably exposes some other
structural issues in the plugin mechanism we're providing to deployers.
[4] https://review.openstack.org/#/c/193401/
[5] https://review.openstack.org/193671
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Angular dependencies and the hz.dashboard namespace

[Richard Jones]

On 20 June 2015 at 09:11, Thai Q Tran tqt...@us.ibm.com wrote:

 -Richard Jones r1chardj0...@gmail.com wrote: -
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 From: Richard Jones r1chardj0...@gmail.com
 Date: 06/19/2015 03:49PM
 Subject: [openstack-dev] Angular dependencies and the hz.dashboard
 namespace


 The Adding Angular Identity Dashboard[1] patch exposed an issue I saw
 previously that worried me[2].

 I believe during recent Horizon work the concept of angular module
 namespaces and dependencies have been conflated. There's this idea that if
 you create a submodule inside a module namespace you *must* have that
 module depend on that submodule. I believe that is incorrect - just look at
 the angular codebase itself, and how it is used. If you want the ngCookies
 module in a couple of places then you have those modules depend on
 ngCookies (or, more likely, you just add it as a dependency to the app).
 The point is it's not just added automatically to the ng module as a
 dependency. If you need to use a module's functionality, you depend on the
 module. You don't just have all your modules *automatically* pulled in as
 dependencies. I have proposed a patch to remedy this for the existing
 optional[3] project dashboard[4].

 I believe it is unnecessary to add extension of the hz.dashboard
 dependency list[5] since we already have an extensible dependency list for
 the application itself (which the hz.dashboard dependency list just
 extends).

 To answer the question explicitly raised what is the point of having the
 hz.dashboard module if it has no dependencies? It does two things: it
 defines the namespace in a formal manner (by itself unnecessary, but it's
 still nice to do) and it defines a constant which is used by other code.
 Eventually it may define more. There is an important difference between
 Python modules and Angular modules here - using a Python module like
 hz.dashboard in this way could cause problems because of the way Python
 sub-module namespaces and import ordering work. Angular's modules work very
 differently, and are not burdened by the same issues. In Python that
 constant would most likely have to be pushed out to a sub-module to avoid
 import loops.


  Richard

 [1] https://review.openstack.org/190852
 [2] https://bugs.launchpad.net/horizon/+bug/1466713
 [3] There may be other issues that prevent the project dashboard being
 optional - there are dependencies in Python-land from the admin dashboard
 hard-coded over to the project dashboard, for example. It might be a good
 idea to remedy that, since I think it probably exposes some other
 structural issues in the plugin mechanism we're providing to deployers.
 [4] https://review.openstack.org/#/c/193401/
 [5] https://review.openstack.org/193671



 1.
 Need to retain the same file structure so that pluggins continue to work.
 Example: https://github.com/stackforge/monasca-ui/tree/master/monitoring

 Basically we have existing pluggins that use this file structure, we need
 to honor it.
 This is not directly related to what we are talking about, but it does
 mean that we need to move static files out of
 /openstack_dashboard/static/MYDASHBOARD/* and into
 /openstack_dashboard/dashboards/MYDASHBOARD/static/*


Yep!



 Auto-discovery of static resources will need to also honor the pluggin
 model above, hence the file structure above.
 You will still need to manually define the ADD_ANGULAR_MODULES in your
 enabled file, auto-discovery doesn't know what you want enabled.
 Sean's patch is going to do that, but having some issues with SCSS.
 https://review.openstack.org/#/c/191592/


As far as I can tell that patch doesn't alter the current ADD_ANGULAR_MODULES
behaviour?



 hz.dashboard module will be empty because the hz.dashboard.MYDASHBOARD
 module will live at the app level via
 ADD_ANGULAR_MODULES. I would argue that it makes no sense to have an empty
 module, my preference is to just delete it.
 Constants are globally available in the app, something I think actually
 should be avoided, not encouraged.


OK, seems reasonable.



 Having hz.dashboard.tech-debt and workflow in the enabled file is not
 correct.
 They are core components needed by all dashboards and should be loaded by
 default, not via the pluggin mechanism.

 https://review.openstack.org/#/c/193401/4/openstack_dashboard/enabled/_10_project.py

 Lets say I have my own dashboard call MYDASHBOARD, and I decided to
 disable all other dashboards except mine,
 all of a sudden, things will break horribly because tech-debt and workflow
 are not loaded. I would have to either:
 a. load the _10_project enabled file
 b. copy/paste over the dependencies from _10_project

 Furthermore, if I have hz.dashboard module, where do I load that, in
 _10_project or _x_MYDASHBOARD?
 Same issue when I disable entire dashboards.


I completely agree!



 Tyr's patch will address this problem by having a core module

Re: [openstack-dev] [javascript] [horizon] [merlin] [refstack] Javascript Linting

[Richard Jones]

JSCS in Horizon has been extended with the John Papa style guidelines to
enforce consistent angularjs code style*. It's no longer just a findbug
tool. I don't have time to investigate - can ESLint perform the same role
for Horizon?

Current Horizon activity involves a whole lot of bringing code into line
with that style (and other JSCS check fails).


  Richard

* https://review.openstack.org/#/c/181311/

On Tue, 16 Jun 2015 at 09:40 Michael Krotscheck krotsch...@gmail.com
wrote:

 I'm restarting this thread with a different subject line to get a broader
 audience. Here's the original thread:
 http://lists.openstack.org/pipermail/openstack-dev/2015-June/066040.html

 The question at hand is What will be OpenStack's javascript equivalent of
 flake8. I'm going to consider the need for common formatting rules to be
 self-evident. Here's the lay of the land so far:

- Horizon currently uses JSCS.
- Refstack uses Eslint.
- Merlin doesn't use anything.
- StoryBoard (deprecated) uses eslint.
- Nobody agrees on rules.

 *JSCS*
 JSCS Stands for JavaScript CodeStyle. Its mission is to enforce a style
 guide, yet it does not check for potential bugs, variable overrides, etc.
 For those tests, the team usually defers to (preferred) JSHint, or ESLint.

 *JSHint*
 Ever since JSCS was extracted from JSHint, it has actively removed rules
 that enforce code style, and focused on findbug style tests instead. JSHint
 still contains the Do no evil license, therefore is not an option for
 OpenStack, and has been disqualified.

 *ESLint*
 ESLint's original mission was to be an OSI compliant replacement for
 JSHint, before the JSCS split. It wants to be a one-tool solution.

 My personal opinion/recommendation: Based on the above, I recommend we use
 ESLint. My reasoning: It's one tool, it's extensible, it does both
 codestyle things and bug finding things, and it has a good license. JSHint
 is disqualified because of the license. JSCS is disqualified because it is
 too focused, and only partially useful on its own.

 I understand that this will mean some work by the Horizon team to bring
 their code in line with a new parser, however I personally consider this to
 be a good thing. If the code is good to begin with, it shouldn't be that
 difficult.

 This thread is not there to argue about which rules to enforce. Right now
 I just want to nail down a tool, so that we can (afterwards) have a
 discussion about which rules to activate.

 Michael
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [javascript] [horizon] [merlin] [refstack] Javascript Linting

[Richard Jones]

Sorry, wrong patch. That one added the style requirement to the project
contribution guidelines. This is the one that added the .jscsrc config:

https://review.openstack.org/#/c/185725/

On Tue, 16 Jun 2015 at 14:21 Richard Jones r1chardj0...@gmail.com wrote:

 JSCS in Horizon has been extended with the John Papa style guidelines to
 enforce consistent angularjs code style*. It's no longer just a findbug
 tool. I don't have time to investigate - can ESLint perform the same role
 for Horizon?

 Current Horizon activity involves a whole lot of bringing code into line
 with that style (and other JSCS check fails).


   Richard

 * https://review.openstack.org/#/c/181311/

 On Tue, 16 Jun 2015 at 09:40 Michael Krotscheck krotsch...@gmail.com
 wrote:

 I'm restarting this thread with a different subject line to get a broader
 audience. Here's the original thread:
 http://lists.openstack.org/pipermail/openstack-dev/2015-June/066040.html

 The question at hand is What will be OpenStack's javascript equivalent
 of flake8. I'm going to consider the need for common formatting rules to
 be self-evident. Here's the lay of the land so far:

- Horizon currently uses JSCS.
- Refstack uses Eslint.
- Merlin doesn't use anything.
- StoryBoard (deprecated) uses eslint.
- Nobody agrees on rules.

 *JSCS*
 JSCS Stands for JavaScript CodeStyle. Its mission is to enforce a style
 guide, yet it does not check for potential bugs, variable overrides, etc.
 For those tests, the team usually defers to (preferred) JSHint, or ESLint.

 *JSHint*
 Ever since JSCS was extracted from JSHint, it has actively removed rules
 that enforce code style, and focused on findbug style tests instead. JSHint
 still contains the Do no evil license, therefore is not an option for
 OpenStack, and has been disqualified.

 *ESLint*
 ESLint's original mission was to be an OSI compliant replacement for
 JSHint, before the JSCS split. It wants to be a one-tool solution.

 My personal opinion/recommendation: Based on the above, I recommend we
 use ESLint. My reasoning: It's one tool, it's extensible, it does both
 codestyle things and bug finding things, and it has a good license. JSHint
 is disqualified because of the license. JSCS is disqualified because it is
 too focused, and only partially useful on its own.

 I understand that this will mean some work by the Horizon team to bring
 their code in line with a new parser, however I personally consider this to
 be a good thing. If the code is good to begin with, it shouldn't be that
 difficult.

 This thread is not there to argue about which rules to enforce. Right now
 I just want to nail down a tool, so that we can (afterwards) have a
 discussion about which rules to activate.

 Michael
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Angular-Django i18n

[Richard Jones]

Hi Thai,

Thanks for writing that up! As I can tell the blueprint only has the
whiteboard which is kinda yuck, so here's my comments:

I'm already on the record for preferring markup approach 3 so we can avoid
having to write our own implementation of a translation engine. I really,
really don't want to see us write our own :/

I'd also like to see us use a minimum-effort approach for extraction and I
think approach 2 is that. It gives is a single point to invoke for message
catalog maintenance and doesn't require deployers to use a node.js tool for
message extraction. I believe Babel (http://babel.pocoo.org/) is the most
appropriate tool already existing for extracting the messages. There's
already a babel plugin for django, we would just need to write one for
angular-gettext. I believe this should allay concerns raised in extraction
approach 2.

And if you're going to do extraction approach 3 then you've practically
implemented a plugin for Babel so you might as well do approach 2 :)

I've looked into extraction approach 4 and got lost in a twisty maze of
ick. I'm not sure there's a clean way to extend Django's makemessage.


 Richard


On Fri, 29 May 2015 at 11:06 Thai Q Tran tqt...@us.ibm.com wrote:

 Hi folks,

 Just drafted a blueprint on this topic so that we can all be on the same
 page. Its quite long, but hopefully you will take some time to read through
 it and provide some meaningful feedback. In the mean time, I will do some
 more investigation and keep you all posted. If you have questions or
 concerns, please contact me in IRC or post it in the comment section of the
 blueprint.

 Link to blueprint:
 https://blueprints.launchpad.net/horizon/+spec/angular-makemessages

 Thanks,
 Thai
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] configuration of plugins in an angularjs world

[Richard Jones]

Hi all,

Just a new conundrum for you :)

At the top level of our base.html dashboard app, we have some dependencies
'horizon.auth', 'horizon.framework' and 'hz.dashboard' (that name will
change to 'horizon.dashboard', but that's for another day). I think we need
to remove 'hz.dashboard' from that list of dependencies and the list needs
to be finer-grained *and configurable*.

Currently in our small dashboard angularjs module structure we have
hz.dashboard depend on all its sub-modules. This means that you can't
disable some dashboards. The JS files would no longer be loaded (because
they are in the enabled structure), but the dependency would remain
(because dependency lists are not).


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] dashboard-app split in horizon

[Richard Jones]

I have now submitted the patch to do the html shenanigans as a new step in
the current ngReorg pipeline (inserted because it should fix a test failure
popping up in the dashboard-app move final step).

https://review.openstack.org/#/c/186295/

Ryan's API reorg should probably depend on this patch also as that should
fix *its* test failures also.

And before anyone says anything, no I'm not particularly thrilled about the
new horizon/test/templates/base.html but frankly I'm not sure how else to
make it work. We could probably cull the JS from that file though. I'm
pretty sure none of the django unit tests exercise JS, and I believe
Selenium works off a different interface (but I've run out of time today to
investigate).


 Richard


On Thu, 28 May 2015 at 02:15 Thai Q Tran tqt...@us.ibm.com wrote:

 Yes Rob, you are correct. ToastService was something Cindy wrote to
 replace horizon.alert (aka messages). We can't remove it because legacy
 still uses it.

 -Rob Cresswell (rcresswe) rcres...@cisco.com wrote: -
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 From: Rob Cresswell (rcresswe) rcres...@cisco.com
 Date: 05/26/2015 11:29PM

 Subject: Re: [openstack-dev] [Horizon] dashboard-app split in horizon

 Went through the files myself and I concur. Most of these files define
 pieces specific to our implementation of the dashboard, so should be moved.

  I’m not entirely sure on where _messages should sit. As we move forward,
 won’t that file just end up as a toast element and nothing more? Maybe
 I’m misinterpreting it, I’m not familiar with toastService.

  Rob


   From: Richard Jones r1chardj0...@gmail.com
 Reply-To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Date: Tuesday, 26 May 2015 01:35
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Cc: Johanson, Tyr H t...@hp.com
 Subject: Re: [openstack-dev] [Horizon] dashboard-app split in horizon

   As a follow-up to this [in the misguided hope that anyone will actually
 read this conversation with myself ;-)] I've started looking at the
 base.html split. At the summit last week, we agreed to:

  1. move base.html over from the framework to the dashboard, and
 2. move the _conf.html and _scripts.html over as well, since they
 configure the application (dashboard).

  Upon starting the work it occurs to me that all of the other files
 referenced by base.html should also move. So, here's the complete list of
 base.html components and whether they should move over in my opinion:

  - horizon/_custom_meta.html
   Yep, is an empty file in horizon, intended as an extension point in
 dashboard. The empty file (plus an added comment) should move.
   - horizon/_stylesheets.html
   Is just a dummy in horizon anyway, should move.
 - horizon/_conf.html
   Yep, should move.
 - horizon/client_side/_script_loader.html
   Looks to be a framework component not intended for override, so we
 should leave it there.
 - horizon/_custom_head_js.html
Yep, is an empty file in horizon, intended as an extension point in
 dashboard. Move, with a comment added.
  - horizon/_header.html
   There is a basic implementation in framework but the real (used)
 implementation is in dashboard, so should move.
 - horizon/_messages.html
   This is a framework component, so I think should stay there. I'm not
 sure whether anyone would ever wish to override this. Also the bulk of it
 is probably going to be replaced by the toast implementation anyway...
 hmm...
 - horizon/common/_sidebar.html
   This is an overridable component that I think should move.
 - horizon/common/_page_header.html
   This is an overridable component that I think should move.
 - horizon/_scripts.html
Yep, should move.

  Thoughts, anyone who has read this far?


  Richard


 On Sat, 23 May 2015 at 11:46 Richard Jones r1chardj0...@gmail.com wrote:

  As part of the ongoing Horizon project code reorganisation, we today
 agreed to clean up the Horizon-the-Framework and OpenStack Dashboard
 separation issue by doing a couple of things:

  1. nuke (the recently-created) horizon dashboard-app by moving the
 angular app over to dashboard and the other contents to appropriate places
 (mostly under the heading of tech-debt :)
 2. move base.html, _conf.html and _scripts.html from horizon over to
 dashboard.

  Thanks to Cindy, Sean and Thai for the pair (er triple?) programming
 keeping me honest today.

  The first step is done and captured in several linked patches based off
 your leaf patch ngReorg - Create dashboard-app 
 https://review.openstack.org/#/c/184597/ (yes, I am nuking the thing
 created by your patch).

  I've not done the second step, but might find some time since I have 6
 hours to waste in LAX tomorrow.


   Richard


 __
 OpenStack Development Mailing List

Re: [openstack-dev] [Horizon] dashboard-app split in horizon

[Richard Jones]

As a follow-up to this [in the misguided hope that anyone will actually
read this conversation with myself ;-)] I've started looking at the
base.html split. At the summit last week, we agreed to:

1. move base.html over from the framework to the dashboard, and
2. move the _conf.html and _scripts.html over as well, since they configure
the application (dashboard).

Upon starting the work it occurs to me that all of the other files
referenced by base.html should also move. So, here's the complete list of
base.html components and whether they should move over in my opinion:

- horizon/_custom_meta.html
  Yep, is an empty file in horizon, intended as an extension point in
dashboard. The empty file (plus an added comment) should move.
  - horizon/_stylesheets.html
  Is just a dummy in horizon anyway, should move.
- horizon/_conf.html
  Yep, should move.
- horizon/client_side/_script_loader.html
  Looks to be a framework component not intended for override, so we should
leave it there.
- horizon/_custom_head_js.html
  Yep, is an empty file in horizon, intended as an extension point in
dashboard. Move, with a comment added.
- horizon/_header.html
  There is a basic implementation in framework but the real (used)
implementation is in dashboard, so should move.
- horizon/_messages.html
  This is a framework component, so I think should stay there. I'm not sure
whether anyone would ever wish to override this. Also the bulk of it is
probably going to be replaced by the toast implementation anyway... hmm...
- horizon/common/_sidebar.html
  This is an overridable component that I think should move.
- horizon/common/_page_header.html
  This is an overridable component that I think should move.
- horizon/_scripts.html
  Yep, should move.

Thoughts, anyone who has read this far?


Richard


On Sat, 23 May 2015 at 11:46 Richard Jones r1chardj0...@gmail.com wrote:

 As part of the ongoing Horizon project code reorganisation, we today
 agreed to clean up the Horizon-the-Framework and OpenStack Dashboard
 separation issue by doing a couple of things:

 1. nuke (the recently-created) horizon dashboard-app by moving the angular
 app over to dashboard and the other contents to appropriate places (mostly
 under the heading of tech-debt :)
 2. move base.html, _conf.html and _scripts.html from horizon over to
 dashboard.

 Thanks to Cindy, Sean and Thai for the pair (er triple?) programming
 keeping me honest today.

 The first step is done and captured in several linked patches based off
 your leaf patch ngReorg - Create dashboard-app 
 https://review.openstack.org/#/c/184597/ (yes, I am nuking the thing
 created by your patch).

 I've not done the second step, but might find some time since I have 6
 hours to waste in LAX tomorrow.


  Richard


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] dashboard-app split in horizon

[Richard Jones]

As part of the ongoing Horizon project code reorganisation, we today agreed
to clean up the Horizon-the-Framework and OpenStack Dashboard separation
issue by doing a couple of things:

1. nuke (the recently-created) horizon dashboard-app by moving the angular
app over to dashboard and the other contents to appropriate places (mostly
under the heading of tech-debt :)
2. move base.html, _conf.html and _scripts.html from horizon over to
dashboard.

Thanks to Cindy, Sean and Thai for the pair (er triple?) programming
keeping me honest today.

The first step is done and captured in several linked patches based off
your leaf patch ngReorg - Create dashboard-app 
https://review.openstack.org/#/c/184597/ (yes, I am nuking the thing
created by your patch).

I've not done the second step, but might find some time since I have 6
hours to waste in LAX tomorrow.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][api] Pagination in thre API

[Richard Jones]

As a consumer of such APIs on the Horizon side, I'm all for consistency in
pagination, and more of it, so yes please!

On 5 November 2015 at 13:24, Tony Breeds  wrote:

> On Thu, Nov 05, 2015 at 01:09:36PM +1100, Tony Breeds wrote:
> > Hi All,
> > Around the middle of October a spec [1] was uploaded to add
> pagination
> > support to the os-hypervisors API.  While I recognize the use case it
> seemed
> > like adding another pagination implementation wasn't an awesome idea.
> >
> > Today I see 3 more requests to add pagination to APIs [2]
> >
> > Perhaps I'm over thinking it but should we do something more strategic
> rather
> > than scattering "add pagination here".
> >
> > It looks to me like we have at least 3 parties interested in this.
> >
> > Yours Tony.
> >
> > [1] https://review.openstack.org/#/c/234038
> > [2]
> https://review.openstack.org/#/q/message:pagination+project:openstack/nova-specs+status:open,n,z
>
> Sorry about the send without complete subject.
>
> Yours Tony.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][policy] Exposing hypervisor details to users

[Richard Jones]

+1 this would definitely be the best approach from Horizon's perspective
since we can cache the capabilities per-flavour. Having to request
capabilities per-instance would be an unreasonable burden on the poor users
of Horizon.

On 6 November 2015 at 23:09, Sean Dague  wrote:

> On 11/06/2015 04:49 AM, Daniel P. Berrange wrote:
> > On Fri, Nov 06, 2015 at 05:08:59PM +1100, Tony Breeds wrote:
> >> Hello all,
> >> I came across [1] which is notionally an ironic bug in that horizon
> presents
> >> VM operations (like suspend) to users.  Clearly these options don't
> make sense
> >> to ironic which can be confusing.
> >>
> >> There is a horizon fix that just disables migrate/suspened and other
> functaions
> >> if the operator sets a flag say ironic is present.  Clealy this is sub
> optimal
> >> for a mixed hv environment.
> >>
> >> The data needed (hpervisor type) is currently avilable only to admins,
> a quick
> >> hack to remove this policy restriction is functional.
> >>
> >> There are a few ways to solve this.
> >>
> >>  1. Change the default from "rule:admin_api" to "" (for
> >> os_compute_api:os-extended-server-attributes and
> >> os_compute_api:os-hypervisors), and set a list of values we're
> >> comfortbale exposing the user (hypervisor_type and
> >> hypervisor_hostname).  So a user can get the hypervisor_name as
> part of
> >> the instance deatils and get the hypervisor_type from the
> >> os-hypervisors.  This would work for horizon but increases the API
> load
> >> on nova and kinda implies that horizon would have to cache the data
> and
> >> open-code assumptions that hypervisor_type can/can't do action $x
> >>
> >>  2. Include the hypervisor_type with the instance data.  This would
> place the
> >> burdon on nova.  It makes the looking up instance details slightly
> more
> >> complex but doesn't result in additional API queries, nor caching
> >> overhead in horizon.  This has the same opencoding issues as Option
> 1.
> >>
> >>  3. Define a service user and have horizon look up the hypervisors
> details via
> >> that role.  Has all the drawbacks as option 1 and I'm struggling to
> >> think of many benefits.
> >>
> >>  4. Create a capabilitioes API of some description, that can be queried
> so that
> >> consumers (horizon) can known
> >>
> >>  5. Some other way for users to know what kind of hypervisor they're
> on, Perhaps
> >> there is an established image property that would work here?
> >>
> >> If we're okay with exposing the hypervisor_type to users, then #2 is
> pretty
> >> quick and easy, and could be done in Mitaka.  Option 4 is probably the
> best
> >> long term solution but I think is best done in 'N' as it needs lots of
> >> discussion.
> >
> > I think that exposing hypervisor_type is very much the *wrong* approach
> > to this problem. The set of allowed actions varies based on much more
> than
> > just the hypervisor_type. The hypervisor version may affect it, as may
> > the hypervisor architecture, and even the version of Nova. If horizon
> > restricted its actions based on hypevisor_type alone, then it is going
> > to inevitably prevent the user from performing otherwise valid actions
> > in a number of scenarios.
> >
> > IMHO, a capabilities based approach is the only viable solution to
> > this kind of problem.
>
> Right, we just had a super long conversation about this in #openstack-qa
> yesterday with mordred, jroll, and deva around what it's going to take
> to get upgrade tests passing with ironic.
>
> Capabilities is the right approach, because it means we're future
> proofing our interface by telling users what they can do, not some
> arbitrary string that they need to cary around a separate library to
> figure those things out.
>
> It seems like capabilities need to exist on flavor, and by proxy instance.
>
> GET /flavors/bm.large/capabilities
>
> {
>  "actions": {
>  'pause': False,
>  'unpause': False,
>  'rebuild': True
>  ..
>   }
>
> A starting point would definitely be the set of actions that you can
> send to the flavor/instance. There may be features beyond that we'd like
> to classify as capabilities, but actions would be a very concrete and
> attainable starting point. With microversions we don't have to solve
> this all at once, start with a concrete thing and move forward.
>
> Sending an action that was "False" for the instance/flavor would return
> a 400 BadRequest high up at the API level, much like input validation
> via jsonschema.
>
> This is nothing new, we've talked about it in the abstract in the Nova
> space for a while. We've yet had anyone really take this on. If you
> wanted to run with a spec and code, it would be welcome.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> 

Re: [openstack-dev] [javascript] [qa] [eslint-config-openstack] Nominating Cindy Lu for eslint-config-openstack-core

[Richard Jones]

+1

On 10 July 2015 at 05:53, David Lyle dkly...@gmail.com wrote:

 On Thu, Jul 9, 2015 at 1:33 PM, Michael Krotscheck krotsch...@gmail.com
 wrote:

 Hey everyone!

 We've been hard at work getting some solid, shared code style rules in
 place for our javascript code, and nobody's been more diligent about this
 than Cindy Lu. Since we've managed to encapsulate our eslint configuration
 into a separate project (much like hacking), I'd like to nominate Cindy Lu
 as a core!

 This will give her the power to tell you whether spaces or tabs are
 better. So be warned :D

 Michael


 +1

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] URL Sanity

[Richard Jones]

Interesting idea, and in general I'm for consistency. I can't speak
directly to the network/port question, though it seems to me that if ports
must be attached to networks then it makes sense for the URL to reflect
that.

On the other hand, some could argue that the django URL routing is ...
legacy ... and shouldn't me messed with :)

On the gripping hand, thinking about this could inform future angular
routing planning...

On 2 September 2015 at 00:39, Rob Cresswell (rcresswe) 
wrote:

> Hi all,
>
> I recently started looking into properly implementing breadcrumbs to make
> navigation clearer, especially around nested resources (Subnets Detail
> page, for example). The idea is to use the request.path to form a logical
> breadcrumb that isn’t dependent on browser history (
> https://review.openstack.org/#/c/129985/3/horizon/browsers/breadcrumb.py ).
> Unfortunately, this breaks down quite quickly because we use odd patterns
> like `//detail`, and `/`
> doesn’t exist.
>
> This made me realise how much of an inconsistent mess the URL patterns
> are.  I’ve started cleaning them up, so we move from these patterns:
>
> `/admin/networks//detail` - Detail page for a Network
> `/admin/networks//addsubnet` - Create page for a Subnet
>
> To patterns in line with usual CRUD usages, such as:
>
> `/admin/networks/` - Detail page for a Network
> `/admin/networks//subnets/create` - Create page for a Subnet
>
> This is mostly trivial, just removing extraneous words and adding
> consistency, with end goal being every panel following patterns like:
>
> `/` - Index page
> `//` - Detail page for a single resource
> `//create` - Create new resource
> `///update` - Update a resource
>
> This gets a little complex around nested items. Should a Port for example,
> which has a unique ID, be reachable in Horizon by just its ID? Ports must
> always be attached to a network as I understand it. There are multiple ways
> to express this:
>
> `/networks/ports/` - Current implementation
> `/networks//ports/` - My preferred structure
> `/ports/` - Another option
>
> Does anyone have any opinions on how to handle this structuring, or if
> it’s even necessary?
>
> Regards,
> Rob
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] python-selenium landed in Debian main today (in Debian Experimental for the moment)

[Richard Jones]

On 9 September 2015 at 05:35, Thomas Goirand  wrote:

> After the non-free files were removed from the package (after I asked
> for it through the Debian bug https://bugs.debian.org/770232), Selenium
> was uploaded and reached Debian Experimental in main today (ie: Selenium
> is not in non-free section of Debian anymore). \o/
>

\o/


Now, I wonder: can the Horizon team use python-selenium as uploaded to
> Debian experimental today? Can we run the Selenium unit tests, even
> without the browser plugins? It is my understanding that it's possible,
> if we use something like PhantomJS, which is also available in Debian.
>

We can't use PhantomJS as a webdriver as a couple of the tests interact
with file inputs and ghostdriver doesn't support those, sadly (and the
developer of ghostdriver is MIA). We are pretty much stuck with just
Firefox as the webdriver.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Horizon Productivity Suggestion

[Richard Jones]

The etherpad that we had running for the bulk of L was really handy;
something like that would be great to keep using to let folks know what is
in play.

On 29 September 2015 at 19:32, Rob Cresswell (rcresswe) 
wrote:

> I wasn’t really envisioning a big discussion on the bugs; more like a
> brief notice period to let reviewers know high-priority items. Could
> definitely spend longer over it if that is preferred. Timing aside, the
> overall idea sounds good though?
>
> Lin: That’s a good idea. A wiki page would probably suffice.
>
> Rob
>
> From: Lin Hua Cheng 
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: Tuesday, 29 September 2015 04:11
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [Horizon] Horizon Productivity Suggestion
>
> I agree with Travis that 2-3 minutes is not enough, that may not be even
> enough to talk about one bug. :)
>
> We could save some time if we have someone monitoring the bugs/feature and
> publish the high priority item into a report - something similar to what
> Keystone does [1].  Reviewers can look this up every time if they need to
> prioritize their reviews.
>
> We can rotate this responsibility among cores every month - even non-core
> if someone wants to volunteer.
>
> -Lin
>
> [1]
> https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting#Keystone_Weekly_Bug_Reports
>
>
>
>
> On Mon, Sep 28, 2015 at 7:22 PM, Tripp, Travis S 
> wrote:
>
>> Things always move more quickly at the end of a cycle because people feel
>> release pressure, but I do think this is a good idea. 2 - 3 minutes isn’t
>> very realistic. It would need to be planned for longer.
>>
>>
>>
>>
>>
>> On 9/28/15, 3:57 AM, "Rob Cresswell (rcresswe)" 
>> wrote:
>>
>> >Hi folks,
>> >
>> >I¹m wondering if we could try marking out a small 2-3 minute slot at the
>> >start of each weekly meeting to highlight Critical/ High bugs that have
>> >code up for review, as well as important blueprints that have code up for
>> >review. These would be blueprints for features that were identified as
>> >high priority at the summit.
>> >
>> >The thought here is that we were very efficient in L-RC1 at moving code
>> >along, which is nice for productivity, but not really great for
>> stability;
>> >it would be good to do this kind of targeted work earlier in the cycle.
>> >I¹ve noticed other projects doing this in their meetings, and it seems
>> >quite effective.
>> >
>> >Rob
>> >
>> >
>>
>> >__
>> >OpenStack Development Mailing List (not for usage questions)
>> >Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Selenium is now green - please pay attention to it

[Richard Jones]

Hi folks,

Selenium tests "gate-horizon-selenium-headless" are now green in master
again.

Please pay attention if it goes red. I will probably notice, but if I
don't, and you can't figure out what's going on, please feel free to get in
touch with me (r1chardj0n3s on IRC in #openstack-horizon, or email). Let's
try to keep it green!


Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] Liberty release - what is the correct version - 2015.2.0, 8.0.0 or 12.0.0?

[Richard Jones]

Hi all,

Historically OpenStack releases are versioned .N as documented in the
Release Naming wiki page[1]. The Liberty Release Schedule[2] has the
version 2015.2.0. However, in Horizon land, I've been informed that
OpenStack is moving to semver. I can't find any information about this move
except a blog post by Doug Hellmann[3]. Is that correct? What is the
version string supposed to be: 8.0.0 as has been used in Horizon's
documentation[4], or 12.0.0 as hinted at by the blog post? I believe that
if we are moving to semver, then 12.0.0 is appropriate.


  Richard

[1] https://wiki.openstack.org/wiki/Release_Naming
[2] https://wiki.openstack.org/wiki/Liberty_Release_Schedule
[3]
https://doughellmann.com/blog/2015/05/29/openstack-server-version-numbering/
[4]
https://raw.githubusercontent.com/openstack/horizon/master/doc/source/topics/settings.rst
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon][release][infra] New xstatic package release proposal

[Richard Jones]

TL;DR: "Releasing xstatic packages" https://review.openstack.org/#/c/253296/

xstatic  is an existing standard for packaging up
static resources for web development (HTML, Javascript and CSS) as Python
modules. Horizon makes use of two dozen xstatic packages.

We need a better-organised method of releasing those packages. Currently
those packages are maintained and released in an ad-hoc manner by
individuals who come and go in the community, with differing release
processes, requiring maintenance handovers, sometimes with people who go
quiet. This does not scale to the dozens of xstatic packages Horizon wishes
to consume.

We therefore wish to use OpenStack's existing release pipeline, handing
over upload to PyPI to that system, while also allowing review of updates
to the packages.

A problem arises in doing so: xstatic packages version themselves using the
three-digit version of the static package they bundle (eg. JQuery-1.2.3)
with an added build number (eg. xstatic-jquery-1.2.3.4) to allow
re-building if the packaging goes wrong. Currently we cannot release these
packages through the OpenStack release pipeline because pbr (a requirement
of that pipeline) does not allow 4 digit version strings. A proposal to
extend pbr to allow 4 digit versions was abandoned today:
https://review.openstack.org/#/c/205623/ (in short, we are only using the
version git tag extraction facility of pbr, and even then not all of that,
and pbr would become more complex with the special-casing required to ...
special-case xstatic packaging).

We will look to use setuptools_scm instead of pbr. Some support work will
be required to make the proposal above happen. This is outlined in
"Releasing xstatic packages" https://review.openstack.org/#/c/253296/

Also, a patch has been submitted to a sample xstatic package implementing
the proposed new workflow: https://review.openstack.org/#/c/252752/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Proposal to add Richard Jones tohorizon-core

[Richard Jones]

Thank you all!

On 9 December 2015 at 04:23, David Lyle <dkly...@gmail.com> wrote:

> Since the results are unanimous, closing early. Welcome Richard!
>
> David
>
> On Thu, Dec 3, 2015 at 12:08 PM, Thai Q Tran <tqt...@us.ibm.com> wrote:
> > An equally BIG +1 from me! Thanks for all the reviews and patches from
> your
> > minions Richard!
> >
> >
> > - Original message -
> > From: David Lyle <dkly...@gmail.com>
> > To: OpenStack Development Mailing List <
> openstack-dev@lists.openstack.org>
> > Cc:
> > Subject: [openstack-dev] [horizon] Proposal to add Richard Jones to
> > horizon-core
> > Date: Wed, Dec 2, 2015 10:57 AM
> >
> > I propose adding Richard Jones[1] to horizon-core.
> >
> > Over the last several cycles Timur has consistently been providing
> > great reviews, actively participating in the Horizon community, and
> > making meaningful contributions around angularJS and overall project
> > stability and health.
> >
> > Please respond with comments, +1s, or objections within one week.
> >
> > Thanks,
> > David
> >
> > [1]
> >
> http://stackalytics.com/?module=horizon-group_id=r1chardj0n3s=all
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Proposal to add Timur Sufiev to horizon-core

[Richard Jones]

+1!

On 3 December 2015 at 07:31, Matthias Runge  wrote:

> On 02/12/15 19:52, David Lyle wrote:
> > I propose adding Timur Sufiev[1] to horizon-core.
> >
> > Over the last several cycles Timur has consistently been providing
> > great reviews, actively participating in the Horizon community, and
> > making meaningful contributions particularly around testing and
> > stability.
> >
> > Please respond with comments, +1s, or objections within one week.
> >
> Oh, yes please! Timur has been doing a great job, not only over the past
> cycle.
>
> Matthias
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] naming of Javascript bits

[Richard Jones]

Yep, thanks folks. Apart from the existing styleguide arguing with my
preferred approach, it seems like we agree on something that should be
pretty consistent. Thanks for putting up that consistency patch Rajat.

On 8 January 2016 at 06:49, Rajat Vig <raj...@thoughtworks.com> wrote:

> I created a patch based off what Richard used as an example to highlight
> inconsistencies and made it consistent.
>
> https://review.openstack.org/#/c/264942/
>
> Have a look and if it feels fine, we can start changing what else exists
> on similar lines.
> Suggestions welcome.
>
> -Rajat
>
> On Thu, Jan 7, 2016 at 10:14 AM Thai Q Tran <tqt...@us.ibm.com> wrote:
>
>> I 2nd that, should rename the file to delete.service.js to match service
>> name.
>>
>>
>> - Original message -
>> From: Rajat Vig <raj...@thoughtworks.com>
>> To: "OpenStack Development Mailing List (not for usage questions)" <
>> openstack-dev@lists.openstack.org>
>> Cc:
>> Subject: Re: [openstack-dev] [Horizon] naming of Javascript bits
>> Date: Thu, Jan 7, 2016 12:03 AM
>>
>> Richard
>>
>> My preference is the same as what you've got there.
>> Fully namespaced Services and Controller allow for better reusability and
>> possibly maintainability.
>> If all "deleteService" were named just that, it'll be mighty confusing to
>> use it in other places.
>>
>> With regards to tying the folder path and the Service/Controller I'd
>> mostly go with that as that encourages simpler rules on how to namespace.
>>
>> For the particular patch you mentioned, the namespaces had a bit of churn
>> which is sort of reflected in what exists in the patch now.
>>
>> If we decide a convention, then we can go and change the bits when the
>> files change next.
>>
>> -Rajat
>>
>>
>>
>> On Wed, Jan 6, 2016 at 10:30 PM Richard Jones <r1chardj0...@gmail.com>
>> wrote:
>>
>> Hi Horizon folks,
>>
>> We've been pretty good about namespacing the new angular code (to the
>> extreme of having a bunch of very similar module files littered around, but
>> that's angular/JS for you, so I'm not going to go on about it ).
>>
>> Anyhoo. One thing I've noticed is that the services, factories and
>> controllers inside those modules aren't being consistently named. We have
>> got a mix of:
>>
>> Launch Instance:
>>
>>   .module('horizon.dashboard.project.workflow.launch-instance')
>>   .factory('launchInstanceModel', launchInstanceModel);
>>
>> The new Images panel:
>>
>>   .module('horizon.app.core.images')
>>   .factory('horizon.app.core.images.row-actions.service', rowActions);
>>
>> and in the same patch:
>>
>>   .module('horizon.app.core.images')
>>   .factory('horizon.app.core.images.actions.deleteService',
>> deleteService);
>>
>> I actually prefer the second form because it matches the filename
>> ("row-actions.service.js") even though the module namespace doesn't match
>> the file path ("/static/app/core/images/table/").
>>
>> Your thoughts?
>>
>>
>>  Richard
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Bug day! Yeah!

[Richard Jones]

Let's do it. I'd like to suggest we use an etherpad to keep track of what
people have done. If it's not created when I start my day, I'll make one.


 Richard

On 19 November 2015 at 22:19, Rob Cresswell (rcresswe) 
wrote:

> Hey folks,
>
> Our bug list is… rather large. We’ve discussed having a bug day, where as
> a community we all dedicate some time to triaging bugs and discussing in
> the IRC channel as we go.
>
> First off, see the docs about bug triage:
> https://wiki.openstack.org/wiki/BugTriage
>
> Secondly, lets pick a date. I’d suggest next Tuesday (24th November); if
> that doesn’t work for a good number of us, then the following Tuesday (1st
> December). Trying to account for Thanksgiving :)
>
> Rob
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Updating XStatic packages

[Richard Jones]

In terms of process, obviously updating the data in the xstatic is
relatively easy (for most packages) but I wonder from a review standpoint
what sort of process we should have here. To approve a merge of an xstatic
package update, I feel like I should have, in order of preference:

1. a link to the change notes for the project in question so I can see what
the impact of the update is, or
2. a list of those changes in the bug or change message for the xstatic
update (i.e. the relevant information from #1), and
3. an explanation of what existing problems are being solved in Horizon
through the update (if any) or what impact there will be on Horizon with
the upgrade (i.e. an examination of #2 in the context of Horizon).

I know that we often need to justify package updates to downstream
packagers, and having this information will certainly help our case. Or is
this all a bit too much burden?


 Richard

On 7 January 2016 at 04:36, Rajat Vig  wrote:

> I did update some of the low hanging packages where the upgrades seemed
> safe to do and noted the patch numbers in the same EtherPad.
>
> I wasn't sure what to mark the effort against. So I created some bugs.
> Should it be a blueprint?
>
> -Rajat
>
> On Wed, Jan 6, 2016 at 2:18 AM, Rob Cresswell (rcresswe) <
> rcres...@cisco.com> wrote:
>
>> Hi all,
>>
>> While the automated system is broken, I’d like to work on manually
>> releasing a few of the XStatic packages. This will *only* be the release
>> stage; we will still use gerrit to review the package content as usual.
>>
>> List of packages current versions, and their upstreams, can be found
>> here: https://etherpad.openstack.org/p/horizon-libs
>>
>> If anyone has spare time, please consider investigating some of the
>> dependencies listed above. To update an XStatic package, propose a change
>> to its repo as you would with Horizon; they are all in the OpenStack
>> namespace. For example, Xstatic-Angular can be found at
>> http://git.openstack.org/cgit/openstack/xstatic-angular/
>>
>> Thanks,
>> Rob
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Routing in Horizon

[Richard Jones]

The  tag addition has had to be reverted as it broke other parts of
the application (notably lazy loaded tabs like Instance Details), sadly.

Regarding which router to use - I've used the built-in router in the past
quite successfully. I think I'd want to see a solid reason for using a 3rd
party one. It could be that nested routes are part of that, but I kinda see
that as a convenience thing, unless I'm missing something core to how
routing is planned to be done. Then again we really haven't had any
discussion about how we see routing work. The one patch that's up that uses
routing seems perfectly able to do so without needing extended capabilities
of 3rd party routers.


 Richard


On 12 January 2016 at 09:44, Tripp, Travis S  wrote:

> Rajat,
>
> Thanks for starting some discussion here.  I think your target=“_self” is
> taken care of now, right?
>
> Re: ng-route vs ui-router - everything I have ever seen or been told
> supports using ui-router instead of ng-route.  I know a quick google search
> supports that, but let me google that for all of us and give several
> references:
>
> http://www.funnyant.com/angularjs-ui-router/
> http://www.amasik.com/angularjs-ngroute-vs-ui-router/
>
> http://stackoverflow.com/questions/21023763/angularjs-difference-between-angular-route-and-angular-ui-router
>
> http://www.pearpages.com/javascript/angularjs/routing/2015/10/13/ngroute-vs-ui-router.html
>
> http://stackoverflow.com/questions/32523512/ui-router-vs-ngroute-for-sinlge-page-app
>
> So, I’m wondering if there’d been any discussion I missed on why not bring
> in ui-router?
>
> Of course there is question using the new router in angular vs ui-router,
> but finding many pros- cons- on that seems to be a bit more difficult.
> Since it is 1.5 / 2.0 and neither are past rc / beta, it doesn’t seem like
> something we can debate too well.
>
> https://angular.github.io/router/getting-started
>
> http://www.angulardaily.com/2015/12/angularintroduction-to-ngnewrouter-vs.html
>
> Thanks,
> Travis
>
> From: Rajat Vig >
> Reply-To: OpenStack List >
> Date: Thursday, January 7, 2016 at 1:53 AM
> To: OpenStack List >
> Subject: [openstack-dev] [Horizon] Routing in Horizon
>
> Hi Everyone
>
> One of my recent patches which enabled HTML5 based routing via Angular
> merged, some interesting things spun out.
> I'd to scramble a few patches to get things
> ​​ back the same way
> for all new Angular Panels.
>
> I also realized that getting Horizon to an SPA with Angular requires more
> thought than mere fixing the current burning issue.
> This mail's intent is to spur a direction on how we do routing in Angular
> and how do Angular Panels go back/forth between themselves and older Django
> panels.
>
> The patch https://review.openstack.org/#/c/173885/ is possibly the first
> of many to use Angular based routing.
> It currently uses ngRoute as the library was included in the
> xstatic-angular package.
>
> What I'm roughly thinking to solve some of the immediate issues (there's
> possbily much more that I'm not)
>
> 1. If we are going to go with the SPA route, then all Panels need to
> indicate that they are Angular based.
> For panels that are Angular, they need to declare routes they'd like to
> manage.
>
> 2. All tags on Angular Panels (including header, sidebar, footer) which
> don't route to Angular Panels will need the attribute target="_self" else
> Angular will not allow navigation to those links.
>
> All sidebar links currently have the attribute set but headers and footers
> don't.
> Sidebar links for Angular shouldn't have the attribute else SPA like
> behavior will not happen.
> This will need to be documen
> ​tation​
> ​
> ​
> ​
> .
>
> 3. That implies yet another problem with the Spinner Modal which gets
> activated on all sidebar clicks.
> It'll need to be done differently for Angular routing vs hrefs still with
> Django.
> The current spinner relies on a browser page refresh to disappear.
>
> Then there's ngRoute.
> Routing needs in Angular may need to handle route conflicts and maybe
> nested routes.
> There are other, possibly better options that we could consider
> 1. https://github.com/angular-ui/ui-router
> 2. https://angular.github.io/router/
>
> I've been part of the community for not long enough yet and I'm not yet
> completely aware of what exists outside the Horizon codebase so I might be
> missing somethings as well.
>
> Regards
> Rajat
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__

[openstack-dev] [Horizon] xstatic publishing update - broken xstatic-angular-bootstrap

[Richard Jones]

Hi folks,

I released a couple of broken xstatic-angular-bootstrap packages to pypi
yesterday - 0.11.0.4 and 0.11.0.5. There was to be an automated release of
0.11.0.6 through openstack-release, but I've abandoned that as it contained
the same breakage. I've performed an emergency direct upload of 0.11.0.7 to
pypi which fixes the issue, and will propose an automated release of
0.11.0.8 some time this weekend.

Future packages should not run into this sort of issue - the root cause is
a bug in the xstatic-release script, which I'm also going to address and
release as 1.1.2 this weekend.


Sorry for the trouble,

  Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] New procedure for releasing xstatic packages

[Richard Jones]

Hi folks,

It's been a lng time coming, but we're almost there. I've put up a BP
outlining the steps we need to get over the line and start releasing those
looong awaited updates of our xstatic packages:

https://blueprints.launchpad.net/horizon/+spec/xstatic-release-process

I'll be working with Tony Breeds and Joshua Hesketh over the next few days
to work through the remaining bits of that BP. Please feel free to review
the BP and the linked changes - in particular the documentation change
which has more detail than the BP:

https://review.openstack.org/#/c/289142

One of the potential gotchas is that we're enforcing that the git tag
version string matches the xstatic package's BUILD version string as part
of the release process (as far as I can tell this is a first for an
OpenStack release process). The downside of this is that you won't know if
it's been rejected unless you sign up to a particular mailing list:

http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures

Well, eventually you'll give up on seeing the package appear in pypi, but
that list is slightly more immediate.


Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Handling 401 in new REST API

[Richard Jones]

Hmm. That client-side redirect isn't working consistently then. More
investigation required...

On 29 January 2016 at 05:04, Thai Q Tran <tqt...@us.ibm.com> wrote:

> I am assuming that you meant redirecting on the server side. We already
> have a similar one in place on the client side.
> https://github.com/openstack/horizon/blob/master/horizon/static/framework/framework.module.js#L65
>
> Currently, you're not able to hit the REST layer directly anyway, You get
> a "request must be ajax" message. You have to login and then use a REST
> client if you want data. So here's my noob question of the day, what are we
> gaining in addition if we redirect on the server side? Totally not meant to
> be a show stopper, just wondering what the benefits are.
>
>
> - Original message -
> From: David Lyle <dkly...@gmail.com>
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Cc:
> Subject: Re: [openstack-dev] [Horizon] Handling 401 in new REST API
> Date: Thu, Jan 28, 2016 8:46 AM
>
> I think that's the sane thing to do.
>
> David
>
> On Thu, Jan 28, 2016 at 2:55 AM, Richard Jones <r1chardj0...@gmail.com>
> wrote:
> > Hi fellow angular/REST Horizon developers,
> >
> > I'd like to propose that we handle HTTP 401 responses at the core of the
> new
> > angular code interfacing to our new REST API so that *any* 401 just does
> > basically what the Django code used to: redirect to the login page with a
> > "from".
> >
> > What do y'all think?
> >
> >
> > Richard
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Handling 401 in new REST API

[Richard Jones]

Hi fellow angular/REST Horizon developers,

I'd like to propose that we handle HTTP 401 responses at the core of the
new angular code interfacing to our new REST API so that *any* 401 just
does basically what the Django code used to: redirect to the login page
with a "from".

What do y'all think?


Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Recent integration tests failures

[Richard Jones]

Ugh, dependencies with breaking API changes in minor point releases :/

On 2 February 2016 at 04:53, Timur Sufiev  wrote:

> Maintainers of outside dependencies continue to break our stuff :(. New
> issue is https://bugs.launchpad.net/horizon/+bug/1540495 patch is
> currently being checked by Jenkins
>
> On Sat, Jan 30, 2016 at 2:28 PM Timur Sufiev  wrote:
>
>> Problematic Selenium versions have been successfully excluded from
>> Horizon test-requirements, if you still experiencing the error described
>> above, rebase your patch onto the latest master.
>> On Fri, 29 Jan 2016 at 12:36, Itxaka Serrano Garcia 
>> wrote:
>>
>>> Can confirm, had the same issue locally, was fixed after a downgrade to
>>> selenium 2.48.
>>>
>>>
>>> Good catch!
>>>
>>> Itxaka
>>>
>>> On 01/28/2016 10:08 PM, Timur Sufiev wrote:
>>> > According to the results at
>>> > https://review.openstack.org/#/c/273697/1 capping Selenium to be not
>>> > greater than 2.49 fixes broken tests. Patch to global-requirements is
>>> > here: https://review.openstack.org/#/c/273750/
>>> >
>>> > On Thu, Jan 28, 2016 at 9:22 PM Timur Sufiev >> > > wrote:
>>> >
>>> > Hello, Horizoneers
>>> >
>>> > You may have noticed recent integration tests failures seemingly
>>> > unrelated to you patches, with a stacktrace like:
>>> > http://paste2.org/2Hk9138U I've already filed a bug for that,
>>> > https://bugs.launchpad.net/horizon/+bug/1539197 Appears to be a
>>> > Selenium issue, currently investigating it.
>>> >
>>> >
>>> >
>>> >
>>> >
>>> __
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Testing of angular promise based code paths

[Richard Jones]

Hi all,

I've just added a comment to a review and I think I'd like to ask for a
broader discussion of whether I'm correct.

The review is here: https://review.openstack.org/#/c/284857/2

It boils down to: when testing code that uses a promise, should we *use* a
promise to have the follow-on callback invoked, or should we mock/spy and
then manually perform the same action the promise would if resolved?

The two forms are, broadly, pretending that a promise fired:

  it('successful submit calls the successCallback', function() {
*var successFunc = {success: angular.noop};*
*spyOn(successFunc, 'success');*
spyOn(nova, 'createKeypair')*.and.returnValue(successFunc);*
spyOn(toastService, 'add').and.returnValue({ add: angular.noop });
ctrl.submit();
*var successCallback = successFunc.success.calls.argsFor(0)[0];*
*var data = {name: 'newKeypair'};*
*successCallback(data);*
expect(toastService.add).toHaveBeenCalledWith(
'success',
'Successfully imported key pair newKeypair.'
);
  });

or actually using a promise and making it fire:

it('should load container contents', function test() {
  *var deferred = $q.defer();*
  spyOn(swiftAPI, 'getObjects')*.and.returnValue(deferred.promise);*
  service.selectContainer('spam');
  expect(service.containerName).toEqual('spam');
  expect(swiftAPI.getObjects).toHaveBeenCalledWith('spam', {delimiter:
'/'});
  *deferred.resolve({data: {items: ['two', 'items']}});*
  *$rootScope.$apply();*
  expect(service.objects).toEqual(['two', 'items']);
  expect(service.pseudo_folder_hierarchy).toEqual([]);
});


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] eslint without color?

[Richard Jones]

I'm just curious why our eslint configuration (in packages.json) specifies
--no-color. It's much harder to spot the errors without color, and I always
end up running it manually to get the color. Also, karma output has color,
so why one and not the other?

In short, would anyone object to turning color on for eslint?


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 10 March 2016 at 18:23, Matthias Runge  wrote:
>
> 4.alt.2:
> move to proper packages for static file management. I mean, they need to
> be built anyways.
>

Please define what you mean by "proper packages" here. I *think* you might
mean system packages (aka Debian or Red Hat) which is not feasible given
other environments that Horizon runs under. Please correct me if I'm wrong!



> It has been mentioned, xstatic packages can block the gate. We currently
> control xstatic package releases, thus we can roll back, if something
> goes wrong.
>
> If we're pulling directly with npm/bower, someone from the outside can
> break us. We already have the situation with pypi packages.
> With proper packages, we could even use the gate to release those
> packages and thus verify, we're not breaking anyone.
>

We're going to have potential breakage (gate breakage, in the integrated
tests) any time we release a package (regardless of release mechanism) and
have to update two separate repositories resulting in out-of-sync version
specification and expectation (ie. upper-constraints specification and
Horizon's code expectation) as described in my OP. The only solution that
we're aware of is to synchronise updating those two things, through one of
the mechanisms proposed so far (or possibly through a mechanism not yet
proposed.)


I think that David's proposal is the only really feasible approach at this
time:

1. Horizon maintains its own constrained version list for the xstatic
packages,
2. Plugins to Horizon must depend on Horizon to supply xstatic packages
except where they use additional packages that Horizon does not use, and
3. We avoid installing app-catalog (the system, not the plugin) in the
integrated tests (I don't believe doing this is even on the ... "horizon"
so to speak) *or* in a Debian / Red Hat (i.e. system-packaged) system that
also has Horizon installed. Or we try to convince app-catalog to stay
lock-step with Horizon's xstatic versions. I understand the risk of a
collision between app-catalog and Horizon in the same system-packaged
environment is very low.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 10 March 2016 at 21:48, Beth Elwell <e.r.elw...@gmail.com> wrote:
>
> On 10 Mar 2016, at 07:46, Richard Jones <r1chardj0...@gmail.com> wrote:
>
>> It has been mentioned, xstatic packages can block the gate. We currently
>> control xstatic package releases, thus we can roll back, if something
>> goes wrong.
>>
>> If we're pulling directly with npm/bower, someone from the outside can
>> break us. We already have the situation with pypi packages.
>> With proper packages, we could even use the gate to release those
>> packages and thus verify, we're not breaking anyone.
>>
>
> We're going to have potential breakage (gate breakage, in the integrated
> tests) any time we release a package (regardless of release mechanism) and
> have to update two separate repositories resulting in out-of-sync version
> specification and expectation (ie. upper-constraints specification and
> Horizon's code expectation) as described in my OP. The only solution that
> we're aware of is to synchronise updating those two things, through one of
> the mechanisms proposed so far (or possibly through a mechanism not yet
> proposed.)
>
> If we will anyway have potential breakage I don’t understand why the
> better solution here would not be to just use the bower and npm tools which
> are standardised for JavaScript and would move Horizon more towards using
> widely recognised tooling from within not just Openstack but the wider
> development community. Back versions always need to be supported for a
> time, however I would add that long term this could end up saving time and
> create a stable longer term solution.
>

I (and others) have argued several times for this, for a number of reasons,
and it remains my preferred solution, but it has been shot down many times
:-(

There are three major hurdles that I recall (sorry if I forgot any, it's
getting late here):

1. system packaging; installers using Debian or Red Hat (completely
offline) installation will not be able to install Horizon. We don't have a
solution for this though various caching mechanisms have been proposed.
2. security; the bower installation mechanism is seen as being very
insecure - not that I would call the current xstatic mechanism secure. It's
all down to degrees, I suppose.
3. installation in the gate; I believe that currently installation from
bower would not be possible in the gate. This is pretty closely related to
#1.

I think I recall licensing came up at one point too, but I might be
mis-remembering.


  Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 13 March 2016 at 07:11, Matthias Runge  wrote:

> On 10/03/16 11:48, Beth Elwell wrote:
> > If we will anyway have potential breakage I don’t understand why the
> > better solution here would not be to just use the bower and npm tools
> > which are standardised for JavaScript and would move Horizon more
> > towards using widely recognised tooling from within not just Openstack
> > but the wider development community. Back versions always need to be
> > supported for a time, however I would add that long term this could end
> > up saving time and create a stable longer term solution.
> >
>
> I have a few issues with those "package managers":
> - downloads are not verified, there is a chance of getting a "bad"
> download.
> - they are pointing to the outside world, like to github etc. While they
> appear to work "most of the time", that might not good enough for the gate
> - how often have we been blocked by releases of software not managed by
> OpenStack? Seriously, that happens quite a few times over a release
> cycle, not to mention breakages by releases of our own tools turning out
> to block one or the other sub-project


To be fair to those package managers,  the issues OpenStack has had with
releases of libraries breaking things is a result of us either:

a) not pinning releases (upper-constraints now fixes that for *things that
use it*, which isn't everything, sadly) or
b) the system that tests upper-constraints changes not having broad enough
testing across OpenStack for us to notice when a new library release breaks
things. I would like to increase the inclusion of Horizon's test suite in
the constraints testing for this reason. At least, it's on my TODO :-)

Horizon, for example, currently does *not* use the upper-constraints
pinning in its test suite or installation, so we're vulnerable to, say, a
python-*client release that's not compatible. I have a patch in the works
to address this, but it kinda depends on us moving over from run_tests.sh
to tox, which is definitely something to wait until N for.


 Richard

ps. as for "unverified downloads" ... they're verified precisely as much as
pypi packages are, and we install a whole buncha those :-)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] PTL noncandidacy

[Richard Jones]

Thanks for everything David. My involvement and engagement with the project
would not be what it is without your leadership. I look forward to
continuing to work with and learn from you!


   Richard

On 12 March 2016 at 04:19, David Lyle  wrote:

> After five cycles as PTL of Horizon, I've decided not to run for the
> Newton cycle.
>
> I am exceptionally proud of the things we've accomplished over this
> time. I'm amazed by how much our project's community has grown and
> evolved.
>
> Looking at the community now, I believe we have a tremendous group of
> contributors for moving forward. There are several people capable of
> becoming great PTLs and overall the community will be healthier with
> some turnover in the PTL role. I feel honored to have had your trust
> over this time and lucky to have worked with such great people.
>
> I will still be around and will help the next PTL make a smooth
> transition where requested.
>
> David
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

But Jeremy: Atomic Zuul. How cool is that name.


So I know I'm going to potentially get yelled at for voicing this, but what
are the chances that app-catalog and Horizon are ever installed on the same
system? That is, could it have its own xstatic constraints, with the
promise that the two constraints will never meet in a dark alley behind
some VM somewhere? app-catalog isn't in the integrated tests, so it's never
going to break the gate like Horizon will...


On 9 March 2016 at 11:07, Jeremy Stanley <fu...@yuggoth.org> wrote:

> On 2016-03-08 17:25:41 +1100 (+1100), Richard Jones wrote:
> [...]
> > SOLUTION 6 - make zuul capable of performing atomic cross-repository
> > commits.
>
> This seems unlikely to happen, as it's very much counter to Zuul's
> designed-in reliance on serializing changes to test before merging
> them.
> --
> Jeremy Stanley
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Javascript linting and documentation

[Richard Jones]

Hey all,

I started looking into fixing the wall of "npm run lint" warnings today and
quickly noticed that about 85% of the "linting" warnings were about jsdoc.
We have significant issues around jsdoc anyway and we're supposed to be
using Sphinx anyway[1].

Some Horizon folks will know that I've been investigating generating
publishable documentation for our Javascript code for some time. Most of
the tools either don't work (dgeni) or produce pretty unusable output for a
project our size (jsdoc and grunt-ngdocs). I am about to investigate Sphinx
in collaboration with the docs team.

Regardless, I believe that the documentation generation should generate
errors about that documentation, not the code linter. Once we actually get
a documentation generator going. Until then, we don't even know what syntax
the documentation should follow.

I've proposed a change which just turns jsdoc "linting" off[2]. At the
moment, it is less than useful (the noise drowns out any other, legitimate
linting).


 Richard


[1] http://governance.openstack.org/reference/cti/javascript-cti.html
[2] https://review.openstack.org/#/c/290235/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Proposal to add Diana Whitten to horizon-core

[Richard Jones]

+1 we need Diana's perspective, knowledge and enthusiasm

On 9 March 2016 at 10:03, Tripp, Travis S  wrote:

> +1, no questions asked. It is rare to find somebody with the passion that
> Diana has shown.
>
> From: Lin Hua Cheng >
> Reply-To: OpenStack List >
> Date: Tuesday, March 8, 2016 at 3:48 PM
> To: OpenStack List >
> Subject: Re: [openstack-dev] [horizon] Proposal to add Diana Whitten to
> horizon-core
>
> big +1 from me.
>
> She made a lot of contribution in making theming better for horizon and
> also prevented potential issues through reviews.
> Thanks for all the hard work, Diana.
>
> -Lin
>
> On Tue, Mar 8, 2016 at 2:06 PM, David Lyle > wrote:
> I propose adding Diana Whitten[1] to horizon-core.
>
> Diana is an active reviewer, contributor and community member. Over
> the past couple of releases, Diana has driven extensive changes around
> theme-ability in Horizon and drastically increased the standardization
> of our use of bootstrap. During this time, Diana has also provided
> valuable reviews to keep us on the straight and narrow preventing our
> continued abuse of defenseless web technologies.
>
> Please respond with comments, +1s, or objections by Friday.
>
> Thanks,
> David
>
> [1]
> http://stackalytics.com/?module=horizon-group_id=hurgleburgler=all
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

Hi Serg,

There's a crucial difference: getting the solution to this wrong for
Horizon will break the gate for all OpenStack projects. Updating an xstatic
package that a plugin uses has no side-effect outside of the plugin.


  Richard

On 10 March 2016 at 02:43, Serg Melikyan  wrote:

> >This is exactly my first thought. The plugins *must* depend on Horizon,
> and they have to use the same versions of XStatic packages anyways, so why
> specify the dependencies twice?
>
> Plugins may require different xstatic library, which is not even used
> by Horizon. Issue raised by Richard exists for plugins too, not only
> for Horizon itself.
>
>
> On Wed, Mar 9, 2016 at 12:24 AM, Radomir Dopieralski
>  wrote:
> > On 03/08/2016 11:43 PM, David Lyle wrote:
> >
> >> I'm wondering if since horizon is really the only project consuming
> >> these xstatic libraries and none are likely to venture down this path of
> >> madness with us that it would be safe to manage the xstatic requirements
> >> and upper-constraints locally.
> >>
> >> Technically the plugins for horizon depend on this, but they depend via
> >> horizon. If they require specific versions that are not supported by
> >> Horizon, I think all bets are off anyway.
> >
> >
> > This is exactly my first thought. The plugins *must* depend on Horizon,
> > and they have to use the same versions of XStatic packages anyways, so
> > why specify the dependencies twice? If the changes between versions are
> > so big as to be breaking, then the plugins have to be updated to work
> > with the new Horizon anyways.
> >
> > --
> > Radomir Dopieralski
> >
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
> http://mirantis.com | smelik...@mirantis.com
>
> +1 (650) 440-8979
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][infra] revert new gerrit

[Richard Jones]

On 19 March 2016 at 02:35, Monty Taylor  wrote:

> On 03/18/2016 08:31 AM, Andrey Kurilin wrote:
>
>> Hi all!
>>
>> I want to start this thread because I'm tired. I spent a lot of time,
>> but I can't review as easy as it was with old interface. New Gerrit is
>> awful. Here are several issues:
>>
>> * It is not possible to review patches at mobile phone. "New" "modern"
>> theme is not adopted for small screens.
>> * Leaving comments is a hard task. Position of page can jump anytime.
>> * It is impossible to turn off hot-keys. position of page changed->I
>> don't see that comment pop-up is closed->continue type several
>> letters->make unexpected things(open edit mode, modify something, save,
>> exit...)
>> * patch-dependency tree is not user-friendly
>> * summary table doesn't include status of patch(I need list to the end
>> of a page to know if patch is merged or not)
>> * there is no button "Comment"/"Reply" at the end of page(after all
>> comments).
>> * it is impossible to turn off "new" search mechanism
>>
>> Does it possible to return old, classic theme? It was a good time when
>> we have old and new themes together...
>>
>
> Sadly no. Upstream is pretty tied to the new very terrible interface.
> We're not sure why.
>
> If you haven't tried gertty yet, I highly recommend it.
>

I hear good things about it but I've tried to get gertty going and found it
doesn't cope well at all with my very low bandwidth Internet connection, so
it's definitely not a solution for everyone :-(

I hold out hope for a usable web ui some day ;-)


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Javascript linting and documentation

[Richard Jones]

On 10 March 2016 at 06:49, Michael Krotscheck  wrote:

> I guess I don't see what problems is being solved by turning the rule off,
> and I also don't see the harm in having more check. It does generate a lot
> of warnings, but invoking `npm run lint -- --quiet` gets rid of those.
>

We already have a "lintq" npm task that does this, which most of us use.
The problem is that we then ignore all the legitimate code linting warnings.


Also, from my experience, sphinx-based doc builds are notoriously
> permissive about bad formatting. Eslint's stricter jsdoc checks would add
> an additional safety net.
>

My perspective on this is if the documentation builder can produce
documentation that is useful then it's enforcing exactly enough checks on
the input. For example, the jsdoc linter currently forces us to write
@returns statements in our docs for angular methods that have no return
value, and never will (i.e. module.config()) or is otherwise not exposed to
developers and not interesting (i.e. the return from service or controller
construction). I mentioned this in passing in the meeting, but most of our
JS documentation has been written to ngdoc syntax, and that's potentially a
good thing since it provides much greater hinting to the doc generators
about how to present and organise the output, but also influences things
like @returns usefulness. We just have to find the right tool (or, more
likely, create, since I've been looking for a while and haven't found a
suitable tool) that uses the information we're coding into our comments.



> However, if you're working on this with the docs team, then the result
> should be applicable to the entirety of openstack - meaning that once your
> work is complete, it may make sense to turn the rule off globally.
>

Yep!


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Javascript linting and documentation

[Richard Jones]

On 11 March 2016 at 04:37, Michael Krotscheck  wrote:

> On Wed, Mar 9, 2016 at 12:45 PM Tripp, Travis S 
> wrote:
>
>> The problem is that the warnings are so great that is really hard to read.
>>
>
> If all the warnings were fixed - I know Matt Borland's working on that -
> would we be having this conversation?
>

My position was that I didn't want Matt to waste his valuable time editing
a bunch of JSDoc when we might end up having to go through and re-edit it
to some other form. We've busy enough with Mitaka release issues during
these weeks without needing to be bothered by fixing ignorable some jsdoc
lint warnings (and they are ignorable - nothing currently cares about the
syntax of our Javascript comments *except* the linter).


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

Thanks, Christopher (and the app-catalog team).

On 21 March 2016 at 08:35, Christopher Aedo <d...@aedo.net> wrote:

> On Sun, Mar 20, 2016 at 1:26 PM, Richard Jones <r1chardj0...@gmail.com>
> wrote:
> > Unfortunately none of this discussion solves the substantive issue which
> is
> > that we cannot release an xstatic package without breaking the gate.
> >
> > We currently have one solution that's a close to viable as we've been
> able
> > to get: move the version pinning for xstatic packages out of
> > upper-constraints and into Horizon's repository, and hope that the
> > app-catalog server and Horizon stay in step or that they're never
> installed
> > on the same debian/redhat system (or if they *are* then one of them is
> > installed in a virtualenv).
>
> The Horizon plugin for the app catalog[1] should have no problem
> staying in step with Horizon.  Ultimately our intention is to make
> this plugin so desirable it becomes a native part of Horizon so
> enabling in a cloud becomes as simple as a config switch.  We're not
> ready for that yet of course, so please don't let that comment
> distract the thread :)  I am looking forward to discussing this with
> more people in Austin though!
>
> Regardless, we will work with you to make sure our efforts proceed to
> be tightly coupled to Horizon.  This approach makes the most sense to
> me.
>
> [1]: http://git.openstack.org/cgit/openstack/app-catalog-ui/
>
> -Christopher
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

Unfortunately none of this discussion solves the substantive issue which is
that we cannot release an xstatic package without breaking the gate.

We currently have one solution that's a close to viable as we've been able
to get: move the version pinning for xstatic packages out of
upper-constraints and into Horizon's repository, and hope that the
app-catalog server and Horizon stay in step or that they're never installed
on the same debian/redhat system (or if they *are* then one of them is
installed in a virtualenv).


 Richard


On 17 March 2016 at 19:00, Thomas Goirand <z...@debian.org> wrote:

> On 03/17/2016 07:23 AM, Richard Jones wrote:
> > There's a basic difference here though. Your traditional "installed
> > components" are pieces of software and data used *by programs on that
> > system.*
> >
> > The components we're talking about here are, as far as the system is
> > concerned, opaque data to be transmitted over HTTP(S) to a web browser
> > client which then makes use of that data in some manner.
> >
> > There are no cross-program compatibility issues stemming from having
> > multiple different versioned copies of such client-side files on a
> > system
>
> The same way, we could have multiple version of fonts, tzdata, SSL root
> certificates and so on. There wouldn't be any compatibility issues.
> Though it's still not the right thing to do at a distribution level.
>
> Have you noticed also that in the Windows world, each program carries
> its .dll, which are supposed to be shared object, but in fact, they
> aren't shared? Yes, it is easier to do so.
>
> > - this is why the web development world has standardised on
> > tooling that *makes it easy to do so*. Different client-side web
> > applications *should* be able to use different versions of components.
>
> The same was as for shared .so libraries, that's not the correct way to
> do things. Even though the JavaScript objects aren't executed by the
> system (well, if we forget that nodejs exists), there's still potential
> bugs and security problems with them, and they may require maintenance.
>
> > xstatic shoe-horns that freedom of client-side application component
> > usage into a one-size-must-fit-all world that fundamentally only exists
> > because programs on a system can get confused when multiple versions are
> > installed on that system[1].
>
> I wouldn't say it this way. To me, they are just tools which makes it
> easy for us to stop the duplication madness of the same files.
>
> Have a look:
> # apt-file search jquery.js | grep -v doc | wc -l
> 128
>
> This is 127 bugs which should be fixed currently with the embedded
> jquery. I hardly see how one could argue this is a good thing. I hope we
> can be better citizens than this.
>
> Cheers,
>
> Thomas Goirand (zigo)
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Richard Jones]

On 13 March 2016 at 07:06, Matthias Runge <mru...@redhat.com> wrote:

> On 10/03/16 08:46, Richard Jones wrote:
> > On 10 March 2016 at 18:23, Matthias Runge <mru...@redhat.com
> > <mailto:mru...@redhat.com>> wrote:
> >
> > 4.alt.2:
> > move to proper packages for static file management. I mean, they
> need to
> > be built anyways.
> >
> >
> > Please define what you mean by "proper packages" here. I *think* you
> > might mean system packages (aka Debian or Red Hat) which is not feasible
> > given other environments that Horizon runs under. Please correct me if
> > I'm wrong!
>
> Exactly. I mean system packages. If there are issues with system
> packages, then let's address the issue rather than re-inventing the wheel.
>

OK, the sticking point is that installation through system packages alone
forces us to make all software on a system work with a single version of
any given library. This has spawned the global-requirements and now
upper-constraints systems in OpenStack, and ultimately leads to the
problematic race condition that resulted in me starting this email thread.
That is, we can update *either* the version of a library being used *or*
the software that is compatible with that version *but not the two at the
same time, atomically*.



> Weren't we just talking about providing dependencies for the gate?


Well, partially, because the reason the problem surfaces is because of the
Continuous Deployment guarantee that OpenStack makes, which is enforced by
the gate, so sure, let's say it's the gate's fault 



> I mean, for production, it's quite the same situation we are at the
> moment. Nobody requires you to install Horizon and dependencies
> especially via rpm, deb or pip: Take what you want.
>

I'm not sure it's this simple, is it? Folks want to be able to install
OpenStack from a single source, and that seems reasonable. They want to be
able to do that "offline" too, so that rules out bower as a source of
packages.



> > It has been mentioned, xstatic packages can block the gate. We
> currently
> > control xstatic package releases, thus we can roll back, if something
> > goes wrong.
> >
> > If we're pulling directly with npm/bower, someone from the outside
> can
> > break us. We already have the situation with pypi packages.
> > With proper packages, we could even use the gate to release those
> > packages and thus verify, we're not breaking anyone.
> >
> >
> > We're going to have potential breakage (gate breakage, in the integrated
> > tests) any time we release a package (regardless of release mechanism)
> > and have to update two separate repositories resulting in out-of-sync
> > version specification and expectation (ie. upper-constraints
> > specification and Horizon's code expectation) as described in my OP. The
> > only solution that we're aware of is to synchronise updating those two
> > things, through one of the mechanisms proposed so far (or possibly
> > through a mechanism not yet proposed.)
> >
>
> Yes, and my proposal to address this is to gate updating/releasing
> dependencies the same way we're currently gating each change in horizon.
>

This is not going to solve the race condition I mention; it's actually
during our work implementing gating these releases that we found we had to
solve this problem.



> > 1. Horizon maintains its own constrained version list for the xstatic
> > packages,
> > 2. Plugins to Horizon must depend on Horizon to supply xstatic packages
> > except where they use additional packages that Horizon does not use, and
> > 3. We avoid installing app-catalog (the system, not the plugin) in the
> > integrated tests (I don't believe doing this is even on the ...
> > "horizon" so to speak) *or* in a Debian / Red Hat (i.e. system-packaged)
> > system that also has Horizon installed. Or we try to convince
> > app-catalog to stay lock-step with Horizon's xstatic versions. I
> > understand the risk of a collision between app-catalog and Horizon in
> > the same system-packaged environment is very low.
>
> I don't really see a chance for app-catalog to require Horizon as a
> dependency and different versions of xstatic packages. This would be an
> immediate show-stopper for app-catalog either on Debian or on RPM based
> systems.
>

I think I used the wrong word above. Where I said "system" I probably
should have said "server". app-catalog the stand-alone server should not
depend on Horizon, just app-catalog the plugin to Horizon should (like all
Horizon plugins should).



> Let me repeat myself: you're free to install dependencies