Re: [openstack-dev] [infra][horizon][osc] ClientImpact tag automation

[Radomir Dopieralski]

To be honest, I don't see much point in automatically creating bugs that
nobody is going to look at. When you implement a new feature, it's up to
you to make it available in Horizon and CLI and wherever else, since the
people working there simply don't have the time to work on it. Creating a
ticket will not magically make someone do that work for you. We are happy
to assist with this, but that's it. Anything else is going to get added
whenever someone has any free cycles, or it becomes necessary for some
reason (like breaking compatibility). That's the current reality, and no
automation is going to help with it.

On Thu, Aug 2, 2018 at 5:09 PM Sean McGinnis  wrote:

> I'm wondering if someone on the infra team can give me some pointers on
> how to
> approach something, and looking for any general feedback as well.
>
> Background
> ==
> We've had things like the DocImpact tag that could be added to commit
> messages
> that would tie into some automation to create a launchpad bug when that
> commit
> merged. While we had a larger docs team and out-of-tree docs, I think this
> really helped us make sure we didn't lose track of needed documentation
> updates.
>
> I was able to find part of how that is implemented in jeepyb:
>
>
> http://git.openstack.org/cgit/openstack-infra/jeepyb/tree/jeepyb/cmd/notify_impact.py
>
> Current Challenge
> =
> Similar to the need to follow up with documentation, I've seen a lot of
> cases
> where projects have added features or made other changes that impact
> downstream
> consumers of that project. Most often, I've seen cases where something like
> python-cinderclient adds some functionality, but it is on projects like
> Horizon
> or python-openstackclient to proactively go out and discover those changes.
>
> Not only just seeking out those changes, but also evaluating whether a
> given
> change should have any impact on their project. So we've ended up in a lot
> of
> cases where either new functionality isn't made available through these
> interfaces until a cycle or two later, or probably worse, cases where
> something
> is now broken with no one aware of it until an actual end user hits a
> problem
> and files a bug.
>
> ClientImpact Plan
> =
> I've run this by a few people and it seems to have some support. Or course
> I'm
> open to any other suggestions.
>
> What I would like to do is add a ClientImpact tag handling that could be
> added
> very similarly to DocImpact. The way I see it working is it would work in
> much
> the same way where project's can use this to add the tag to a commit
> message
> when they know it is something that will require additional work in OSC or
> Horizon (or others). Then when that commit merges, automation would create
> a
> launchpad bug and/or Storyboard story, including a default set of client
> projects. Perhaps we can find some way to make those impacted clients
> configurable by source project, but that could be a follow-on optimization.
>
> I am concerned that this could create some extra overhead for these
> projects.
> But my hope is it would be a quick evaluation by a bug triager in those
> projects where they can, hopefully, quickly determine if a change does not
> in
> fact impact them and just close the ones they don't think require any
> follow on
> work.
>
> I do hope that this will save some time and speed things up overall for
> these
> projects to be notified that there is something that needs their attention
> without needing someone to take the time to actively go out and discover
> that.
>
> Help Needed
> ===
> From the bits I've found for the DocImpact handling, it looks like it
> should
> not be too much effort to implement the logic to handle a ClientImpact
> flag.
> But I have not been able to find all the moving parts that work together to
> perform that automation.
>
> If anyone has any background knowledge on how DocImpact is implemented and
> can
> give me a few pointers, I think I should be able to take it from there to
> get
> this implemented. Or if there is someone that knows this well and is
> interested
> in working on some of the implementation, that would be very welcome too!
>
> Sean
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Release-job-failures][release][horizon] Release of openstack/xstatic-angular-material failed

[Radomir Dopieralski]

A fix for it is in review: https://review.openstack.org/577820

On Mon, Jun 25, 2018 at 3:07 PM, Andreas Jaeger  wrote:

> On 2018-06-25 14:57, Radomir Dopieralski wrote:
> > Any idea where it took the 1.1.5.0 version from?
>
> git grep 1.1.5 shows at least:
>
> setup.cfg:description = Angular-Material 1.1.5 (XStatic packaging standard)
> xstatic/pkg/angular_material/__init__.py:VERSION = '1.1.5'  # version of
> the packaged files, please use the upstream
>
> Not sure whether that's the right place, I suggest you build locally and
> check the build tarball,
>
> Andreas
>
> > On Mon, Jun 25, 2018 at 2:38 PM, Doug Hellmann  > <mailto:d...@doughellmann.com>> wrote:
> >
> > Excerpts from zuul's message of 2018-06-25 12:14:34 +:
> > > Build failed.
> > >
> > > - xstatic-check-version
> > http://logs.openstack.org/59/592a9d4c90f37cd33c6f861120f41a
> c8a67d909b/release/xstatic-check-version/a501dba/
> > <http://logs.openstack.org/59/592a9d4c90f37cd33c6f861120f41a
> c8a67d909b/release/xstatic-check-version/a501dba/>
> > : FAILURE in 1m 31s
> > > - release-openstack-python release-openstack-python : SKIPPED
> > > - announce-release announce-release : SKIPPED
> > > - propose-update-constraints propose-update-constraints : SKIPPED
> > >
> >
> > "git tag version (1.1.5.1) does not match package version (1.1.5.0)"
> >
> > http://logs.openstack.org/59/592a9d4c90f37cd33c6f861120f41a
> c8a67d909b/release/xstatic-check-version/a501dba/job-
> output.txt.gz#_2018-06-25_12_14_11_034599
> > <http://logs.openstack.org/59/592a9d4c90f37cd33c6f861120f41a
> c8a67d909b/release/xstatic-check-version/a501dba/job-
> output.txt.gz#_2018-06-25_12_14_11_034599>
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > <http://openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >
> >
> >
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
> --
>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Release-job-failures][release][horizon] Release of openstack/xstatic-angular-material failed

[Radomir Dopieralski]

Any idea where it took the 1.1.5.0 version from?

On Mon, Jun 25, 2018 at 2:38 PM, Doug Hellmann 
wrote:

> Excerpts from zuul's message of 2018-06-25 12:14:34 +:
> > Build failed.
> >
> > - xstatic-check-version http://logs.openstack.org/59/
> 592a9d4c90f37cd33c6f861120f41ac8a67d909b/release/xstatic-
> check-version/a501dba/ : FAILURE in 1m 31s
> > - release-openstack-python release-openstack-python : SKIPPED
> > - announce-release announce-release : SKIPPED
> > - propose-update-constraints propose-update-constraints : SKIPPED
> >
>
> "git tag version (1.1.5.1) does not match package version (1.1.5.0)"
>
> http://logs.openstack.org/59/592a9d4c90f37cd33c6f861120f41a
> c8a67d909b/release/xstatic-check-version/a501dba/job-
> output.txt.gz#_2018-06-25_12_14_11_034599
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] [heat-dashboard] Horizon plugin settings for new xstatic modules

[Radomir Dopieralski]

to understand what
>>>>> is
>>>>> >> >>> xstatic
>>>>> >> >>> and how it is maintained.
>>>>> >> >>>
>>>>> >> >>> Thanks,
>>>>> >> >>> Akihiro
>>>>> >> >>>
>>>>> >> >>>
>>>>> >> >>> 2018-03-20 17:17 GMT+09:00 Kaz Shinohara >>>> >:
>>>>> >> >>>>
>>>>> >> >>>> Hi Akihiro,
>>>>> >> >>>>
>>>>> >> >>>>
>>>>> >> >>>> Thanks for your comment.
>>>>> >> >>>> The background of my request to add us to xstatic-core comes
>>>>> from
>>>>> >> >>>> Ivan's comment in last PTG's etherpad for heat-dashboard
>>>>> discussion.
>>>>> >> >>>>
>>>>> >> >>>> https://etherpad.openstack.org/p/heat-dashboard-ptg-
>>>>> rocky-discussion
>>>>> >> >>>> Line135, "we can share ownership if needed - e0ne"
>>>>> >> >>>>
>>>>> >> >>>> Just in case, could you guys confirm unified opinion on this
>>>>> matter
>>>>> >> >>>> as
>>>>> >> >>>> Horizon team ?
>>>>> >> >>>>
>>>>> >> >>>> Frankly speaking I'm feeling the benefit to make us
>>>>> xstatic-core
>>>>> >> >>>> because it's easier & smoother to manage what we are taking for
>>>>> >> >>>> heat-dashboard.
>>>>> >> >>>> On the other hand, I can understand what Akihiro you are
>>>>> saying, the
>>>>> >> >>>> newly added repos belong to Horizon project & being managed by
>>>>> not
>>>>> >> >>>> Horizon core is not consistent.
>>>>> >> >>>> Also having exception might make unexpected confusion in near
>>>>> future.
>>>>> >> >>>>
>>>>> >> >>>> Eventually we will follow your opinion, let me hear Horizon
>>>>> team's
>>>>> >> >>>> conclusion.
>>>>> >> >>>>
>>>>> >> >>>> Regards,
>>>>> >> >>>> Kaz
>>>>> >> >>>>
>>>>> >> >>>>
>>>>> >> >>>> 2018-03-20 12:58 GMT+09:00 Akihiro Motoki :
>>>>> >> >>>> > Hi Kaz,
>>>>> >> >>>> >
>>>>> >> >>>> > These repositories are under horizon project. It looks
>>>>> better to
>>>>> >> >>>> > keep
>>>>> >> >>>> > the
>>>>> >> >>>> > current core team.
>>>>> >> >>>> > It potentially brings some confusion if we treat some horizon
>>>>> >> >>>> > plugin
>>>>> >> >>>> > team
>>>>> >> >>>> > specially.
>>>>> >> >>>> > Reviewing xstatic repos would be a small burden, wo I think
>>>>> it
>>>>> >> >>>> > would
>>>>> >> >>>> > work
>>>>> >> >>>> > without problem even if only horizon-core can approve xstatic
>>>>> >> >>>> > reviews.
>>>>> >> >>>> >
>>>>> >> >>>> >
>>>>> >> >>>> > 2018-03-20 10:02 GMT+09:00 Kaz Shinohara <
>>>>> ksnhr.t...@gmail.com>:
>>>>> >> >>>> >>
>>>>> >> >>>> >> Hi Ivan, Horizon folks,
>>>>> >> >>>> >>
>>>>> >> >>>> >>
>>>>> >> >>>> >> Now totally 8 xstatic-** repos for heat-dashboard have been
>>>>> >> >>>> >> landed.
>>>>> >> >>>> >>
&g

Re: [openstack-dev] [horizon] Font awesome currently broken with Debian Sid and Horizon

[Radomir Dopieralski]

I did a quick search for all the glyphs we are using:

~/dev/horizon(master)> ag 'fa-' | egrep -o 'fa-[a-z-]*' | sort | uniq
fa-
fa-angle-left
fa-angle-right
fa-arrow-down
fa-arrow-up
fa-asterisk
fa-b
fa-bars
fa-bolt
fa-bug
fa-calculator
fa-calendar
fa-caret-down
fa-caret-up
fa-check
fa-chevron-down
fa-chevron-right
fa-circle
fa-close
fa-cloud
fa-cloud-upload
fa-code
fa-cog
fa-desktop
fa-download
fa-exclamation
fa-exclamation-circle
fa-exclamation-triangle
fa-eye
fa-eye-slash
fa-font-path
fa-fw
fa-group
fa-home
fa-icon
fa-info-circle
fa-lg
fa-list-alt
fa-lock
fa-minus
fa-pencil
fa-plus
fa-question-circle
fa-refresh
fa-save
fa-search
fa-server
fa-share-square-o
fa-sign-out
fa-sort
fa-spin
fa-spinner
fa-square
fa-th
fa-th-large
fa-times
fa-trash
fa-unlock
fa-upload
fa-user
fa-var-check-square-o
fa-var-circle-o
fa-var-dot-circle-o
fa-var-sort
fa-var-sort-asc
fa-var-sort-desc
fa-var-square-o
fa-warning

The lone "fa-" comes from table actions, where the icon is defined in code,
and pretty much anything can be used —
since plugins can define their own actions with their own icons — but I
don't think we use anything exotic there.

On Sat, May 26, 2018 at 7:37 PM, Thomas Goirand  wrote:

> Hi Horizon team!
>
> I'm not sure if you're aware of that, but the upstream authors of
> fontawesome decided it was a good idea to break everyone. See this
> Debian bug entry:
>
> https://bugs.debian.org/899124
>
> So, what happened is that fontawesome-webfont has been split into 3 sets
> of fonts: solid, regular and brands fonts. Thus there is no drop in
> replacement for the old fontawesome-webfont.xxx.
>
> As my python3-xstatic-font-awesome removes the embedded fonts, and just
> points to /usr/share/fonts-font-awesome, Horizon is broken and cannot
> even be installed currently in Debian Sid. Of course, I'm considering
> reverting the removal of the data folder from the xstatic package, but
> it then defeats the purpose of Xstatic, which is avoiding duplication of
> static files already packaged in the distribution.
>
> So, ideally, I would like to know first if I can use the fa-solid-900
> for Horizon, or if other glyphs are in use (it's very much possible that
> only fa-solid-900 stuff are in use, but I really don't know how to check
> for that the correct way). If that's the case, then I can workaround the
> issue (at least temporarily), and synlink stuff in the data folder to
> the new fa-solid-900 files.
>
> Second, it'd be nice if Horizon could adapt and use the new v5
> font-awesome, so that the problem is completely solved.
>
> Cheers,
>
> Thomas Goirand (zigo)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon][xstatic]How to handle xstatic if upstream files are modified

[Radomir Dopieralski]

The whole idea about xstatic files is that they are generic, not specific
to Horizon or OpenStack, usable by other projects that need those static
files. In fact, at the time we started using xstatic, it was being used by
the MoinMoin wiki project (which is now dead, sadly). The modifications you
made are very specific to your usecase and would make it impossible to
reuse the packages by other applications (or even by other Horizon
plugins). The whole idea of a library is that you are using it as it is
provided, and not modifying it.

We generally try to use all the libraries as they are, and if there are any
modifications necessary, we push them upstream, to the original library.
Otherwise there would be quite a bit of maintenance overhead necessary to
keep all our downstream patches. When considerable modification is
necessary that can't be pushed upstream, we fork the library either into
its own repository, or include it in the repository of the application that
is using it.

On Mon, Apr 9, 2018 at 2:54 AM, Xinni Ge  wrote:

> Hello, team.
>
> Sorry for talking about xstatic repo for so many times.
>
> I didn't realize xstatic repositories should be provided with exactly the
> same file as upstream, and should have talked about it at very first.
>
> I modified several upstream files because some of them files couldn't be
> used directly under my expectation.
>
> For example,  {{ }} are used in some original files as template tags, but
> Horizon adopts {$ $} in angular module, so I modified them to be recognized
> properly.
>
> Another major modification is that css files are converted into scss files
> to solve some css import issue previously.
> Besides, after collecting statics, some png file paths in css cannot be
> referenced properly and shown as 404 errors, I also modified css itself to
> handle this issues.
>
> I will recheck all the un-matched xstatic repositories and try to replace
> with upstream  files as much as I can.
> But I if I really have to modify some original files, is it acceptable to
> still use it as embedded files with license info appeared at the top?
>
>
> Best Regards,
> Xinni Ge
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] [heat-dashboard] Horizon plugin settings for new xstatic modules

[Radomir Dopieralski]

Yes, please do that. We can then discuss in the review about technical
details.

On Mon, Mar 12, 2018 at 2:54 AM, Xinni Ge  wrote:

> Hi, Akihiro
>
> Thanks for the quick reply.
>
> I agree with your opinion that BASE_XSTATIC_MODULES should not be
> modified.
> It is much better to enhance horizon plugin settings,
>  and I think maybe there could be one option like ADD_XSTATIC_MODULES.
> This option adds the plugin's xstatic files in STATICFILES_DIRS.
> I am considering to add a bug report to describe it at first, and give a
> patch later maybe.
> Is that ok with the Horizon team?
>
> Best Regards.
> Xinni
>
> On Fri, Mar 9, 2018 at 11:47 PM, Akihiro Motoki  wrote:
>
>> Hi Xinni,
>>
>> 2018-03-09 12:05 GMT+09:00 Xinni Ge :
>> > Hello Horizon Team,
>> >
>> > I would like to hear about your opinions about how to add new xstatic
>> > modules to horizon settings.
>> >
>> > As for Heat-dashboard project embedded 3rd-party files issue, thanks for
>> > your advices in Dublin PTG, we are now removing them and referencing as
>> new
>> > xstatic-* libs.
>>
>> Thanks for moving this forward.
>>
>> > So we installed the new xstatic files (not uploaded as openstack
>> official
>> > repos yet) in our development environment now, but hesitate to decide
>> how to
>> > add the new installed xstatic lib path to STATICFILES_DIRS in
>> > openstack_dashboard.settings so that the static files could be
>> automatically
>> > collected by *collectstatic* process.
>> >
>> > Currently Horizon defines BASE_XSTATIC_MODULES in
>> > openstack_dashboard/utils/settings.py and the relevant static fils are
>> added
>> > to STATICFILES_DIRS before it updates any Horizon plugin dashboard.
>> > We may want new plugin setting keywords ( something similar to
>> ADD_JS_FILES)
>> > to update horizon XSTATIC_MODULES (or directly update STATICFILES_DIRS).
>>
>> IMHO it is better to allow horizon plugins to add xstatic modules
>> through horizon plugin settings. I don't think it is a good idea to
>> add a new entry in BASE_XSTATIC_MODULES based on horizon plugin
>> usages. It makes difficult to track why and where a xstatic module in
>> BASE_XSTATIC_MODULES is used.
>> Multiple horizon plugins can add a same entry, so horizon code to
>> handle plugin settings should merge multiple entries to a single one
>> hopefully.
>> My vote is to enhance the horizon plugin settings.
>>
>> Akihiro
>>
>> >
>> > Looking forward to hearing any suggestions from you guys, and
>> > Best Regards,
>> >
>> > Xinni Ge
>> >
>> > 
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: openstack-dev-requ...@lists.op
>> enstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> 葛馨霓 Xinni Ge
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Approach for bugs in xstatic packages

[Radomir Dopieralski]

I can see several possible ways forward with this:

* provide a complete fix for the issue that would be more likely to get
merged upstream,
* instead of adding hacks in the code, actually rename the wrongly named
files,
* instead of adding hacks, actually fix all the references to use proper
capitalization,
* if everything else fails, look for a better supported project that
provides this functionality.



On Tue, Jun 13, 2017 at 11:25 PM, Mateusz Kowalski  wrote:

> Hi everyone,
>
> I would like to raise a question about our approach to xstatic packages,
> more specifically xstatic-roboto-fontface, and its updates/bugfixes. What
> we have encountered was https://bugs.launchpad.net/horizon/+bug/1671004 what
> according to our knowledge affects everyone using stable/ocata with
> Material design (though I get there may not be many operators using this
> setup).
>
> What I did at the very first was to submit a patch https://review.
> openstack.org/#/c/443025/ but unfortunately a policy is to take xstatic
> packages from the upstream in their unchanged form. As I totally understand
> this, I raised this issue to the upstream package provider in March —
> https://github.com/choffmeister/roboto-fontface-bower/issues/41. Again,
> unfortunately, as since then there was no any response I submitted a merge
> request for this issue — https://github.com/choffmeister/roboto-fontface-
> bower/pull/47. However, it’s complete openstack-wise, but it does not
> solve the issue in general (long story short, more than just one file
> requires patching, but Horizon uses only the one I have patched).
>
> Anyway, seeing no much response for my initial upstream report since
> March, I don’t have much hope with the merge request I have just submitted.
> From the other side, any advice on how I can proceed in this particular
> case would be helpful as I have no any experience in this kind of workflow
> (bugs in xstatic packages).
>
> Thanks for any suggestions,
> Mateusz
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Do we need service types at all?! (Re: [octavia][sdk] service name for octavia)

[Radomir Dopieralski]

I think that you have to remember that OpenStack doesn't only work with
officially approved OpenStack services, but with any services that have a
conforming API. OpenStack itself provides implementations of those
services, and you are right that it's unlikely that there will be two
competing implementations for the same service type officially endorsed by
OpenStack. But you are ignoring 3rd-party services that are being used, and
also the possibility that the services will change in the future. Since it
doesn't really cost us to keep this flexibility, I think it's reasonable to
keep doing it.

On Thu, Feb 16, 2017 at 11:55 AM, Andrey Kurilin 
wrote:

> Hi everyone!
>
> When I started contribution to OpenStack long time ago, I asked about the
> reason of having two entities - service type and name; at that time I got
> an answer that service name is a name of the project that implements
> particular service type, so it is possible to have several projects which
> implement one service type. That answer satisfied me.
>
> The reality of OpenStack is that there is no and there will not be
> several implementations of one "service type". Even when we had nova-net
> and neutron, only the neutron registered his service in the catalog.
>
> An example of Octavia shows that everybody was ok about having service
> name equal with type before inconsistency was found. That is why I want to
> ask again: Do we need two separate entities and if yes - why? Maybe service
> name and description fields should be enough?
>
> On Wed, Feb 15, 2017 at 3:08 AM, Qiming Teng 
> wrote:
>
>> When reviewing a recent patch that adds openstacksdk support to octavia,
>> I found that octavia is using 'octavia' as its service name instead of
>> 'loadbalancing' or 'loadbalancingv2' or something similar.
>>
>> The overall suggestion is to use a word/phrase that indicates what a
>> service do instead of the name of the project providing that service.
>>
>> Below is the list of the service types currently supported by
>> openstacksdk:
>>
>> 'alarming',# aodh
>> 'baremetal',   # ironic
>> 'clustering',  # senlin
>> 'compute', # nova
>> 'database',# trove
>> 'identity',# keystone
>> 'image',   # glance
>> 'key-manager', # barbican
>> 'messaging',   # zaqar
>> 'metering',# ceilometer
>> 'network', # neutron
>> 'object-store',   # swift
>> 'octavia',# <--- this is an exception
>> 'orchestration',  # heat
>> 'volume', # cinder
>> 'workflowv2', # mistral
>>
>> While I believe this has been discussed about a year ago, I'm not sure
>> if there are things we missed so I'm brining this issue to a broader
>> audience for discussion.
>>
>> Reference:
>>
>> [1] Patch to python-openstacksdk:
>> https://review.openstack.org/#/c/428414
>> [2] Octavia service naming:
>> http://git.openstack.org/cgit/openstack/octavia/tree/devstac
>> k/plugin.sh#n52
>>
>> Regards,
>>  Qiming
>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Best regards,
> Andrey Kurilin.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] feature freeze exception request -- nova simple tenant usages api pagination

[Radomir Dopieralski]

I prepared a patch that forces the use of 2.40 api version if it's
available for that particular endpoint. Instead of waiting for the
microversion patch, I simply copied the needed fragments from it -- we can
fix that later once we have proper microversion support implemented.

https://review.openstack.org/#/c/424585/

On Tue, Jan 24, 2017 at 4:08 PM, Diana Clarke <diana.joan.cla...@gmail.com>
wrote:

> It was done like that to circumvent the existing possibility DoS-like
> usage requests when there are thousands of instances.
>
> Some of the history behind that decision can be found in the spec
> discussions [1].
>
> And an easier to read spec can be found here [2].
>
> [1] https://review.openstack.org/#/c/386771/
> [2] https://specs.openstack.org/openstack/nova-specs/specs/
> ocata/approved/paginate-simple-tenant-usage.html
>
> --diana
>
> On Tue, Jan 24, 2017 at 4:11 AM, Radomir Dopieralski
> <openst...@sheep.art.pl> wrote:
> > No, for some reason Nova will now always limit the number of entries it
> > sends in a single response, no matter what microversion you use. If you
> use
> > microversion of at least 2.40, it will let you request more responses, to
> > get all the entries. I don't know why they did it like that.
> >
> > On Tue, Jan 24, 2017 at 9:52 AM, Rob Cresswell (rcresswe)
> > <rcres...@cisco.com> wrote:
> >>
> >> As I understand it, if someone configures Nova to use 2.40 via settings,
> >> then it will use 2.40 for every request. This could likely break
> Horizon in
> >> weird ways, which makes it seem risky to try and support it.
> >>
> >> What I don’t really understand about this FFE, is why we need to modify
> >> the behaviour at all; if we keep using an old microversion (I think it
> >> defaults to 2.1?) then shouldn’t behaviour stay exactly the same?
> >>
> >> Rob
> >>
> >> > On 23 Jan 2017, at 21:08, Richard Jones <r1chardj0...@gmail.com>
> wrote:
> >> >
> >> > [I'm on vacation, so can't look into this too deeply, sorry]
> >> >
> >> > I'm not sure I follow Rob's point here. Does the patch
> >> > https://review.openstack.org/#/c/410337 just check the version to see
> >> > if it's >= 2.40 and take action appropriately? I don't see how it
> >> > changes anything to force requesting 2.40 with every request? Then
> >> > again, I've not been able to look into how the current clients'
> >> > microversion code is implemented/broken. Is it just that *declaring*
> >> > the 2.40 version in https://review.openstack.org/#/c/422642 results
> in
> >> > all requests being forced to use that version?
> >> >
> >> >
> >> > Richard
> >> >
> >> > On 23 January 2017 at 23:10, Radomir Dopieralski
> >> > <openst...@sheep.art.pl> wrote:
> >> >> Yes, to do it differently we need to add the microversion support
> patch
> >> >> that
> >> >> you are working on, and make use of it, or write a patch that has
> >> >> equivalent
> >> >> functionality.
> >> >>
> >> >> On Fri, Jan 20, 2017 at 6:57 PM, Rob Cresswell
> >> >> <robert.cressw...@outlook.com> wrote:
> >> >>>
> >> >>> Just a thought: With the way we currently do microversions, wouldnt
> >> >>> this
> >> >>> request 2.40 for every request ? There's a pretty good chance that
> >> >>> would
> >> >>> break things.
> >> >>>
> >> >>> Rob
> >> >>>
> >> >>> On 20 January 2017 at 00:02, Richard Jones <r1chardj0...@gmail.com>
> >> >>> wrote:
> >> >>>>
> >> >>>> FFE granted for the three patches. We need to support that nova API
> >> >>>> change.
> >> >>>>
> >> >>>> On 20 January 2017 at 01:28, Radomir Dopieralski
> >> >>>> <openst...@sheep.art.pl>
> >> >>>> wrote:
> >> >>>>> I would like to request a feature freeze exception for the
> following
> >> >>>>> patch:
> >> >>>>>
> >> >>>>> https://review.openstack.org/#/c/410337
> >> >>>>>
> >> >>>>> This patch adds support for retrieving the simple tenant usages
> from
> >> >>>>> Nova in
> >> >>>>&

Re: [openstack-dev] [horizon] feature freeze exception request -- nova simple tenant usages api pagination

[Radomir Dopieralski]

No, for some reason Nova will now always limit the number of entries it
sends in a single response, no matter what microversion you use. If you use
microversion of at least 2.40, it will let you request more responses, to
get all the entries. I don't know why they did it like that.

On Tue, Jan 24, 2017 at 9:52 AM, Rob Cresswell (rcresswe) <
rcres...@cisco.com> wrote:

> As I understand it, if someone configures Nova to use 2.40 via settings,
> then it will use 2.40 for every request. This could likely break Horizon in
> weird ways, which makes it seem risky to try and support it.
>
> What I don’t really understand about this FFE, is why we need to modify
> the behaviour at all; if we keep using an old microversion (I think it
> defaults to 2.1?) then shouldn’t behaviour stay exactly the same?
>
> Rob
>
> > On 23 Jan 2017, at 21:08, Richard Jones <r1chardj0...@gmail.com> wrote:
> >
> > [I'm on vacation, so can't look into this too deeply, sorry]
> >
> > I'm not sure I follow Rob's point here. Does the patch
> > https://review.openstack.org/#/c/410337 just check the version to see
> > if it's >= 2.40 and take action appropriately? I don't see how it
> > changes anything to force requesting 2.40 with every request? Then
> > again, I've not been able to look into how the current clients'
> > microversion code is implemented/broken. Is it just that *declaring*
> > the 2.40 version in https://review.openstack.org/#/c/422642 results in
> > all requests being forced to use that version?
> >
> >
> > Richard
> >
> > On 23 January 2017 at 23:10, Radomir Dopieralski <openst...@sheep.art.pl>
> wrote:
> >> Yes, to do it differently we need to add the microversion support patch
> that
> >> you are working on, and make use of it, or write a patch that has
> equivalent
> >> functionality.
> >>
> >> On Fri, Jan 20, 2017 at 6:57 PM, Rob Cresswell
> >> <robert.cressw...@outlook.com> wrote:
> >>>
> >>> Just a thought: With the way we currently do microversions, wouldnt
> this
> >>> request 2.40 for every request ? There's a pretty good chance that
> would
> >>> break things.
> >>>
> >>> Rob
> >>>
> >>> On 20 January 2017 at 00:02, Richard Jones <r1chardj0...@gmail.com>
> wrote:
> >>>>
> >>>> FFE granted for the three patches. We need to support that nova API
> >>>> change.
> >>>>
> >>>> On 20 January 2017 at 01:28, Radomir Dopieralski <
> openst...@sheep.art.pl>
> >>>> wrote:
> >>>>> I would like to request a feature freeze exception for the following
> >>>>> patch:
> >>>>>
> >>>>> https://review.openstack.org/#/c/410337
> >>>>>
> >>>>> This patch adds support for retrieving the simple tenant usages from
> >>>>> Nova in
> >>>>> chunks, and it is necessary for correct data given that related
> patches
> >>>>> have
> >>>>> been already merged in Nova. Without
> >>>>> it, the data received will be truncated.
> >>>>>
> >>>>> In order to actually use that patch, however, it is necessary to set
> >>>>> the
> >>>>> Nova API version to at least
> >>>>> version 3.40. For this, it's necessary to also add this patch:
> >>>>>
> >>>>> https://review.openstack.org/422642
> >>>>>
> >>>>> However, that patch will not work, because of a bug in the
> >>>>> VersionManager,
> >>>>> which for some reason
> >>>>> uses floating point numbers for specifying versions, and thus
> >>>>> understands
> >>>>> 2.40 as 2.4. To fix that, it
> >>>>> is also necessary to merge this patch:
> >>>>>
> >>>>> https://review.openstack.org/#/c/410688
> >>>>>
> >>>>> I would like to request an exception for all those three patches.
> >>>>>
> >>>>> An alternative to this would be to finish and merge the microversion
> >>>>> support, and modify the first patch to make use of it. Then we would
> >>>>> need
> >>>>> exceptions for those two patches.
> >>>>>
> >>>>>
> >>>>> 
> __
> &

Re: [openstack-dev] [horizon] feature freeze exception request -- nova simple tenant usages api pagination

[Radomir Dopieralski]

Yes, to do it differently we need to add the microversion support patch
that you are working on, and make use of it, or write a patch that has
equivalent functionality.

On Fri, Jan 20, 2017 at 6:57 PM, Rob Cresswell <robert.cressw...@outlook.com
> wrote:

> Just a thought: With the way we currently do microversions, wouldnt this
> request 2.40 for every request ? There's a pretty good chance that would
> break things.
>
> Rob
>
> On 20 January 2017 at 00:02, Richard Jones <r1chardj0...@gmail.com> wrote:
>
>> FFE granted for the three patches. We need to support that nova API
>> change.
>>
>> On 20 January 2017 at 01:28, Radomir Dopieralski <openst...@sheep.art.pl>
>> wrote:
>> > I would like to request a feature freeze exception for the following
>> patch:
>> >
>> > https://review.openstack.org/#/c/410337
>> >
>> > This patch adds support for retrieving the simple tenant usages from
>> Nova in
>> > chunks, and it is necessary for correct data given that related patches
>> have
>> > been already merged in Nova. Without
>> > it, the data received will be truncated.
>> >
>> > In order to actually use that patch, however, it is necessary to set the
>> > Nova API version to at least
>> > version 3.40. For this, it's necessary to also add this patch:
>> >
>> > https://review.openstack.org/422642
>> >
>> > However, that patch will not work, because of a bug in the
>> VersionManager,
>> > which for some reason
>> > uses floating point numbers for specifying versions, and thus
>> understands
>> > 2.40 as 2.4. To fix that, it
>> > is also necessary to merge this patch:
>> >
>> > https://review.openstack.org/#/c/410688
>> >
>> > I would like to request an exception for all those three patches.
>> >
>> > An alternative to this would be to finish and merge the microversion
>> > support, and modify the first patch to make use of it. Then we would
>> need
>> > exceptions for those two patches.
>> >
>> > 
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: openstack-dev-requ...@lists.op
>> enstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] feature freeze exception request -- nova simple tenant usages api pagination

[Radomir Dopieralski]

I would like to request a feature freeze exception for the following patch:

https://review.openstack.org/#/c/410337

This patch adds support for retrieving the simple tenant usages from Nova
in chunks, and it is necessary for correct data given that related patches
have been already merged in Nova. Without
it, the data received will be truncated.

In order to actually use that patch, however, it is necessary to set the
Nova API version to at least
version 3.40. For this, it's necessary to also add this patch:

https://review.openstack.org/422642

However, that patch will not work, because of a bug in the VersionManager,
which for some reason
uses floating point numbers for specifying versions, and thus understands
2.40 as 2.4. To fix that, it
is also necessary to merge this patch:

https://review.openstack.org/#/c/410688

I would like to request an exception for all those three patches.

An alternative to this would be to finish and merge the microversion
support, and modify the first patch to make use of it. Then we would need
exceptions for those two patches.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Using \ for multiline statements

[Radomir Dopieralski]

There is a problem with the use of backslash at the end of the line, where
if you also put a space after it, it no longer does what it seemingly
should. Together with the fact, that you can achieve the same thing using
() in pretty much all instances (it used to not be true for imports, but
that has been fixed), it suggests that it should be avoided. However, it
does help to format certain Java-esque constructs (mostly with mox) in a
more readable way, so sometimes it's justified.

On Fri, Dec 23, 2016 at 12:28 AM, Sean McGinnis 
wrote:

> Looking for input from everyone, particularly those with more in-depth
> Python knowledge.
>
> In Cinder for some time we have been trying to enforce using () or
> reformatting code to avoid using \ to have statements span multiple
> lines. I'm not sure when this actually started, but I think it may
> be one of those things where someone got a review disagreement, so
> then that person started downvoting on it, then the next person, etc.
>
> I've seen some allusions to the use of \ having some issues, but I
> can't find any concrete examples where this can cause problems. I do
> seem to remember trying to write a hacking check or a code parsing
> tool to do something that choked on these, but it's long enough ago
> that I don't remember the details, and I could very well be mixing
> that up with something else.
>
> So my question is - is there a technical reason for enforcing this
> rule, or is this just a bad downvote that's gotten out of control?
>
> Thanks!
>
> Sean
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Weekly wrap-up

[Radomir Dopieralski]

I wonder if it really makes sense to put WIP patches on the priority list.
I think it's a bit counter-productive, considering that the prioritizing of
patches was supposed to make them merge faster -- but we don't want to
merge WIP patches, do we?

On Fri, Dec 16, 2016 at 5:31 AM, Richard Jones 
wrote:

> Hi folks,
>
> No Horizon meeting next week! I'll be around the week after (28th
> December) so if anyone else is around we can totally have a meeting
> then.
>
> Things that have happened this week, including in the team meeting[1]
>
> Ocata-2 was tagged!
>
> xstatic-angular-bootstrap 2.2.0.0 was released, which promptly broke
> Ocata-2 (and Ocata-1, you're welcome). We knew it was coming, and Rob
> Cresswell had a compatibility patch in place ready to go, so master
> was back to working within half an hour of the upper-constraints patch
> enabling 2.2.0.0! If you notice anything busted in Horizon related to
> bootstrap please file a bug!
>
> I'd like to remind everyone that we have a blueprint in play which
> describes our approach to API microversions[2]. Rob has a patch which
> will land imminently that implements the core of the design. Please
> hold off implementing your own version settings/checks! We've seen a
> few microversion-related patches appear recently, and that's great to
> see, we just need to make sure we're all heading in the same
> direction.
>
> I've put up the first patch using ui-router which rather dramatically
> alters the Swift UI[3], so I'd like some feedback on it please.
>
> We've got about five(ish) weeks until Feature Freeze[4], folks, so all
> the help we can get on the priority patches[5] (reviews or coding
> help) is appreciated!
>
>
> I hope you all get to have some time off, and have an enjoyable holiday,
>
>Richard
>
> [1] http://eavesdrop.openstack.org/meetings/horizon/2016/
> horizon.2016-12-14-20.00.html
> [2] https://blueprints.launchpad.net/horizon/+spec/microversion-support
> [3] https://review.openstack.org/#/c/350523
> [4] https://releases.openstack.org/ocata/schedule.html
> [5] https://review.openstack.org/#/q/starredby:r1chardj0n3s%
> 20AND%20status:open
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Draft team mascot

[Radomir Dopieralski]

Here, fixed.

On Wed, Dec 7, 2016 at 10:54 AM, Radomir Dopieralski <openst...@sheep.art.pl
> wrote:

> That looks kinda like a white baboon. It definitely doesn't look like Doge
> -- wrong color, wrong head. I think the legs are too long too.
>
> On Wed, Dec 7, 2016 at 10:31 AM, Timur Sufiev <tsuf...@mirantis.com>
> wrote:
>
>> I still think this one https://wtf.jpg.wtf/0c/10/
>> 1479414543-0c1052f7c2f9990b6b0c472076594cb1.jpeg is the best :).
>>
>> On Wed, Dec 7, 2016 at 1:07 AM Jason Rist <jr...@redhat.com> wrote:
>>
>>> On 12/06/2016 01:48 PM, Richard Jones wrote:
>>> > >> On 6 Dec 2016, at 20:19, Richard Jones <r1chardj0...@gmail.com>
>>> wrote:
>>> > >> Please let me know what you think (by December 12) of this draft for
>>> > >> our Horizon team mascot.
>>> > >
>>> > On 7 December 2016 at 07:38, Rob Cresswell (rcresswe)
>>> > <rcres...@cisco.com> wrote:
>>> > > Are we missing an attachment / link ?
>>> >
>>> > Weird! Trying again:
>>> >
>>> >
>>> >
>>> > 
>>> __
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe: openstack-dev-requ...@lists.op
>>> enstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>> Much UI, such OpenStack, wow.
>>>
>>> --
>>> Jason E. Rist
>>> Senior Software Engineer
>>> OpenStack User Interfaces
>>> Red Hat, Inc.
>>> Freenode: jrist
>>> github/twitter: knowncitizen
>>>
>>> 
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.op
>>> enstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Draft team mascot

[Radomir Dopieralski]

That looks kinda like a white baboon. It definitely doesn't look like Doge
-- wrong color, wrong head. I think the legs are too long too.

On Wed, Dec 7, 2016 at 10:31 AM, Timur Sufiev  wrote:

> I still think this one https://wtf.jpg.wtf/0c/10/1479414543-
> 0c1052f7c2f9990b6b0c472076594cb1.jpeg is the best :).
>
> On Wed, Dec 7, 2016 at 1:07 AM Jason Rist  wrote:
>
>> On 12/06/2016 01:48 PM, Richard Jones wrote:
>> > >> On 6 Dec 2016, at 20:19, Richard Jones 
>> wrote:
>> > >> Please let me know what you think (by December 12) of this draft for
>> > >> our Horizon team mascot.
>> > >
>> > On 7 December 2016 at 07:38, Rob Cresswell (rcresswe)
>> >  wrote:
>> > > Are we missing an attachment / link ?
>> >
>> > Weird! Trying again:
>> >
>> >
>> >
>> > 
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
>> unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> Much UI, such OpenStack, wow.
>>
>> --
>> Jason E. Rist
>> Senior Software Engineer
>> OpenStack User Interfaces
>> Red Hat, Inc.
>> Freenode: jrist
>> github/twitter: knowncitizen
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
>> unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Proposing Kenji Ishii for core

[Radomir Dopieralski]

+1

On Mon, Nov 14, 2016 at 1:24 AM, Richard Jones 
wrote:

> Hi Horizon core team,
>
> I propose Kenji Ishii as a new Horizon core reviewer. Kenji has been a
> solid Horizon contributor for some time, with thoughtful and helpful
> reviews showing good judgment and good knowledge of Horizion and
> related systems. Please respond with your votes by Friday.
>
>
> Thanks,
>
> Richard
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Browser Support

[Radomir Dopieralski]

On Tue, Sep 20, 2016 at 3:53 AM, Jason Rist  wrote:

> This page hasn't been updated for a while - does anyone know the latest?
>
> https://wiki.openstack.org/wiki/Horizon/BrowserSupport
>
>
As far as I know, there were no changes to the officially supported browser
versions.

The support for very old browsers (like msie 6) is going to deteriorate
slowly, as the libraries that we use for styles and js drop support for
them.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OPENSTACK] [HORIZON] Changing theme

[Radomir Dopieralski]

Did you forget to run collectstatic?

On Fri, Jul 15, 2016 at 10:00 AM, BIKRAMADITYA SAHOO <
ansh.ansh20...@gmail.com> wrote:

> Hi,
>
> I am unable to change the theme for my openstack horizon. I tried with the
> this  and
> various other tutorial but i am still unable to achieve my goal.
>
> I also want to have a custom "splash" (a complete custom login) and i am
> unable to find any good tutorial.
>
> I tried "local_settings.py" with following
> CUSTOM_THEME_PATH="static/themes/production"
> CUSTOM_THEME_PATH="themes/production"
> CUSTOM_THEME_PATH="static/production"
>
> "production" directory is present in
> "/usr/share/openstack-dashboard/openstack_dashboard/static/themes" as well
> as "/usr/share/openstack-dashboard/openstack_dashboard/static/custom"
> directories.
>
> Also i tried deleting the "i/usr/share/openstack-dashboard-ubuntu-theme"
> and
> "/usr/share/openstack-dashboard/openstack_dashboard/local/ubuntu_theme.py"
> and still no use.
>
> No matter what i do, horizon is loading images from "static/themes/ubuntu
> " directory ("
> http://controller/static/themes/ubuntu/image-background-pattern.png;).
>
> Kindly point me to anything i am doing wrong or/and any well-documented
> tutorial or docs.
>
> NOTE: I want to have the above changes in an already running setup by
> manipulating files.
>
> Thanks and Regards,
> Bikramaditya Sahoo
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla][horizon] Out of branch horizon plugins?

[Radomir Dopieralski]

Hi,

sorry for late reply. The whole "openstack_dashboard/local/enabled/"
mechanism was inspired by the mechanisms used commonly in Debian and other
distributions to enable/disable plugins. In short, you shouldn't *copy* the
configuration files to the "enabled" directory -- instead you should keep
them with the plugin, and only symlink them there when you actually want
the plugin to be enabled. There is no need for additional logic in the
plugin checking any environment variables and so on.

On Tue, Jul 5, 2016 at 7:57 PM, Fox, Kevin M  wrote:

> I wrote the app-catalog-ui plugin. I was going to bring this up but hadn't
> gotten to it yet. Thanks for bringing it up.
>
> We do package it up in an rpm, so if its installed with the rest of the
> packages it should just work. The horizon compress/collect rpm hook does
> the right thing already. It does cause it to be enabled though, so I was
> thinking, maybe we make a docker environment variable ENABLED_PLUGINS that
> contains a list of plugins to be enabled?
>
> The value of ENABLED_PLUGINS could be written into
> /etc/openstack-dashboard/local-settings.d and the plugins could enable
> themselves based on it?
>
> I'd rather have the horizon container be bigger then needed and have all
> the plugins test/ready to go as needed instead of trying to slide the
> plugins in myself. Its kind of a pain.
>
> Thanks,
> Kevin
> --
> *From:* Dave Walker [em...@daviey.com]
> *Sent:* Sunday, July 03, 2016 1:15 PM
> *To:* OpenStack Development Mailing List
> *Subject:* [openstack-dev] [kolla][horizon] Out of branch horizon plugins?
>
> Hi,
>
> Whilst writing a Kolla plugin, I noticed some issues with the way Horizon
> is configured in Kolla.
>
> Horizon is increasingly embracing a plugin architecture with UI's and
> Dashboards being maintained outside of Horizon's tree.
>
> These can be found with the type:horizon-plugin tag in
> openstack/governance [0], with 16 projects at current.
>
> This isn't really addressed in Kolla, and is a little awkward to integrate
> as the horizon docker image is pure horizon.
>
> Some projects have a tools/register_plugin.sh which performs the grunt
> work, where as others require a workflow similar to:
>
> cp /path/to/projects/new/panel openstack_dashboard/local/enabled/
> cp /path/to/local/defualt/settings
> openstack_dashboard/local/local_settings.d/
> cp /path/to/*policy.json openstack_dashboard/conf/
> # compress if environment wants it
> ./manage.py collectstatic
> ./manage.py compress
>
> (Separately, it would be nice if this was standardised.. but not the topic
> of this thread)
>
> It would seem logical to pack all of these into the horizon docker image,
> and add a symlink into dashboard/local/enabled via ansible, policy.json and
> default settings with some conditionals if enabled_$service... but this
> will make the horizon docker image larger and more complicated.
>
> What are your thoughts?
>
> [0]
> http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml
>
> --
> Kind Regards,
> Dave Walker
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Radomir Dopieralski]

On 03/09/2016 04:43 PM, Serg Melikyan wrote:

This is exactly my first thought. The plugins *must* depend on Horizon,
and they have to use the same versions of XStatic packages anyways,

>> so why specify the dependencies twice?


Plugins may require different xstatic library, which is not even used
by Horizon. Issue raised by Richard exists for plugins too, not only
for Horizon itself.


How would such an xstatic library conflict with what is in Horizon then,
though?

--
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] patches that improve code quality

[Radomir Dopieralski]

Some of the code we have in Nova is admittedly hard to read and brittle. 
This is true even though we cave excellent code review,
and the reviewers that deeply care for the quality of code that is being 
merged. Some of this code has been there always, some got that way as a 
result of bug fixes and patches that were intended to touch as little 
code as possible, and some is just to bad judgment or mistakes.


Whatever the reason, such code accumulates and makes the program harder 
to understand, to debug and to modify. The maintenance and development 
costs rise. It's also less pleasant to work with such code base.


Cleaning up such code and rewriting it to be easier to understand and 
more robust is a good practice that every programmer should be familiar 
with. However, it's not always easy or even possible to do that while 
working on a feature or a bugfix. If our patch includes a lot of changes 
that are unrelated to the feature or bug at hand, it becomes harder to 
understand and review, and even runs a risk of being rejected. It's also 
easier to introduce bugs that way.


A good practice would be to make such changes in separate patches, 
possibly also adding tests (where they are missing) showing that there 
is no change in behavior. Such patches can be reviewed with special 
attention to readability and robustness, and in the review there will be 
often even further improvements. We end up with a better program and 
easier (and more pleasant) work.


And now, finally, I can get to the point of this e-mail. I'm relatively 
new to this project, but I found no way to direct the (precious) 
attention of core reviewers to such patches. They are not bugs, neither 
they are parts of any new feature, and so there is no list in Nova that 
core reviewers look at where we could add such a patch. Admittedly, such 
patches are not urgent -- the code works the same (or almost the same) 
way without them -- but it would be nice to have some way of merging 
them eventually, because they do make our lives easier in the long run.


So here's my question: what is the correct way to have such a patch 
merged in Nova, and -- if there isn't one -- should we think about 
making it easier?

--
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] How do we move forward with xstatic releases?

[Radomir Dopieralski]

On 03/08/2016 11:43 PM, David Lyle wrote:


I'm wondering if since horizon is really the only project consuming
these xstatic libraries and none are likely to venture down this path of
madness with us that it would be safe to manage the xstatic requirements
and upper-constraints locally.

Technically the plugins for horizon depend on this, but they depend via
horizon. If they require specific versions that are not supported by
Horizon, I think all bets are off anyway.


This is exactly my first thought. The plugins *must* depend on Horizon,
and they have to use the same versions of XStatic packages anyways, so
why specify the dependencies twice? If the changes between versions are
so big as to be breaking, then the plugins have to be updated to work
with the new Horizon anyways.

--
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

[Radomir Dopieralski]

On 01/08/2016 09:51 PM, Mike Bayer wrote:



On 01/08/2016 04:44 AM, Radomir Dopieralski wrote:

On 01/07/2016 05:55 PM, Mike Bayer wrote:


but also even if you're under something like
mod_wsgi, you can spawn a child process or worker thread regardless.
You always have a Python interpreter running and all the things it can
do.


Actually you can't, reliably. Or, more precisely, you really shouldn't.
Most web servers out there expect to do their own process/thread
management and get really embarrassed if you do something like this,
resulting in weird stuff happening.


I have to disagree with this as an across-the-board rule, partially
because my own work in building an enhanced database connection
management system is probably going to require that a background thread
be running in order to reap stale database connections.   Web servers
certainly do their own process/thread management, but a thoughtfully
organized background thread in conjunction with a supporting HTTP
service allows this to be feasible.   In the case of mod_wsgi,
particularly when using mod_wsgi in daemon mode, spawning of threads,
processes and in some scenarios even wholly separate applications are
supported use cases.


[...]


It is certainly reasonable that not all web application containers would
be effective with apps that include custom background threads or
processes (even though IMO any system that's running a Python
interpreter shouldn't have any issues with a limited number of
well-behaved daemon-mode threads), but at least in the case of mod_wsgi,
this is supported; that gives Openstack's HTTP-related applications with
carefully/thoughtfully organized background threads at least one
industry-standard alternative besides being forever welded to its
current homegrown WSGI server implementation.


This is still writing your application for a specific configuration of a 
specific version of a specific implementation of the protocol on a 
specific web server. While this may work as a stopgap solution, I think
it's a really bad long-term strategy. We should be programming for a 
protocol specification (WSGI in this case), not for a particular 
implementation (unless we need to throw in workarounds for 
implementation bugs). This way of thinking led to the trouble we have 
right now, and the fix is not to change the code to exploit another 
specific implementation, but to rewrite it so that it works on any 
compatible web server. If possible.


At least it seems so to my naive programmer mind. Sorry for ranting,
I'm sure that you are aware of the trade-off here.

--
Radomir Dopieralski

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

[Radomir Dopieralski]

On 01/07/2016 05:55 PM, Mike Bayer wrote:


but also even if you're under something like
mod_wsgi, you can spawn a child process or worker thread regardless.
You always have a Python interpreter running and all the things it can do.


Actually you can't, reliably. Or, more precisely, you really shouldn't. 
Most web servers out there expect to do their own process/thread 
management and get really embarrassed if you do something like this,

resulting in weird stuff happening.

--
Radomir Dopieralski

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Core Reviewer Update

[Radomir Dopieralski]

On 04/29/2015 12:57 AM, David Lyle wrote:
 I am pleased to announce the addition of Doug Fish, Rob Cresswell and
 Travis Tripp to the Horizon Core Reviewer team. 
 
 Doug Fish has been an active reviewer and participant in Horizon for a
 few releases now. He represents a strong customer focus and has provided
 high quality reviews. 
 
 Rob Cresswell has been providing a high number of quality reviews, an
 active contributor and an active participant in the community.
 
 Travis Tripp has been contributing to Horizon for the past couple of
 releases, an active participant in the community, a critical angularJS
 reviewer, and played a significant role in driving the angular based
 launch instance work in Kilo.  
 
 Thank you all for your contributions and welcome to the team!

Welcome!

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Do No Evil

[Radomir Dopieralski]

On 03/09/2015 01:59 PM, Michael Krotscheck wrote:
 On Sun, Mar 8, 2015 at 3:21 PM Thomas Goirand z...@debian.org
 mailto:z...@debian.org wrote:
 
 
 Anyway, you understood me: please *never* use this Expat/MIT license
 with the The Software shall be used for Good, not Evil. additional
 clause. This is non-free software, which I will *never* be able to
 upload to Debian (and Canonical guys will have the same issue).
 
 
 So, to clarify: Does this include tooling used to build the software,
 but is not shipped with it? I suppose a similar example is using GCC
 (which is GPL'd) to compile something that's Apache licensed.

To clarify, we are not shipping jshint/jslint with horizon, or requiring
you to have it in order to run or build horizon. It's not used in the
build process, the install process or at runtime. The only places where
it is used is at the developer's own machine, when they install it and
run it explicitly, to check their code, and on the gate, to check the
code submitted for merging. In either case we are not distributing any
software, so no copyright applies.

One could argue that since the review process often causes a lot of
stress both to the authors of the patches and to their reviewers, and so
in fact we are using the software for evil...

We are working on switching to ESLint, not strictly because of the
license, but simply because it seems to be a better and more flexible
tool, but this is not very urgent, and will likely take some time.

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] - Add custom JS functions to dashboard

[Radomir Dopieralski]

On 16/02/15 14:54, Marcos Fermin Lobo wrote:
 Hi all,
 
 I would like to add some (own) JavaScript functions to the image list,
 in Project dashboard.
 
 I've followed up the documentation
 (http://docs.openstack.org/developer/horizon/topics/customizing.html#custom-javascript),
 but I think it is outdated, because that documentation refers to
 directory tree which is not the same in (for example) Juno release. I
 mean, there is no:
 openstack_dashboard/dashboards/project/templates/project/ (check
 https://github.com/cernops/horizon/tree/master-patches/openstack_dashboard/dashboards/project)
 
 
 So, my question is: Where should I write my own JavaScript functions to
 be able to use them in the image list (project dashboard)?. The
 important point here is that my new JavaScript functions should be
 available to compress process which is execute (by default) during RPM
 building.
 
 Thank you for your time.

That documentation is still valid, even if the example paths are
different. We use exactly this technique in tuskar-ui and it still works.

Starting with Juno, you can also add JavaScript files in your panel's
configuration file, see
http://docs.openstack.org/developer/horizon/topics/settings.html#add-js-files

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] JavaScript docs?

[Radomir Dopieralski]

On 02/05/2015 07:26 PM, Michael Krotscheck wrote:
 On Thu Feb 05 2015 at 12:07:01 AM Radomir Dopieralski
 openst...@sheep.art.pl mailto:openst...@sheep.art.pl wrote:
 
 
 Plus, the documentation generator that we are using already, Sphinx,
 supports JavaScript perfectly fine, so I see no reason to add
 another tool.
 
 
 Try to empathize with us a little here. What you're asking is equivalent
 to saying OpenStack should use JavaDoc for all its documentation
 because it supports python. For all the reasons that you would mistrust
 JavaDoc, I mistrust Sphinx when it comes to parsing javascript.
 
 With that in mind, how about we run a side-by-side comparison of Sphinx
 vs. NGDoc? Without actual comparable output, this discussion isn't much
 more than warring opinions.

I'm not mistrusting JavaDoc or NGDoc or whatever new documentation
system you are proposing. I merely think that, while JavaScript
programmers are special snowflakes indeed, they are not special enough
warrant introducing and maintaining a whole separate documentation
system, especially since we are already using a documentation system
that is well used and maintained by the whole of OpenStack, not just the
Python programmers in Horizon. And since you will have to learn to use
Sphinx sooner or later anyways, because basically everything in
OpenStack is documented using it, I see no reason why we should expend
additional energy on implementing, deploying and maintaining a new set
of tools, just because you don't like the current one.

If it was JavaDoc instead of Sphinx being used by the whole of
OpenStack, I would advocate its use the same way as I advocate Sphinx now.

It seems that the whole docs format discussion is just a way of putting
away having to actually write the docs.

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] JavaScript docs?

[Radomir Dopieralski]

On 02/05/2015 06:20 PM, Matthew Farina wrote:
 I'd like to step back for a moment as to the purpose of different kinds
 of documentation. Sphinx is great and it provides some forms of
 documentation. But, why do we document methods, classes, or functions in
 python? Should we drop that and rely on Sphinx? I don't think anyone
 would argue for that.

We actually rely on Sphinx for documenting methods, classes or
functions. Not sure what your point is here.

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] JavaScript docs?

[Radomir Dopieralski]

On 02/04/2015 06:06 PM, Thai Q Tran wrote:
 As we're moving toward Angular, might make sense for us to adopt ngdoc
 as well.

I don't think it makes much sense. We don't have any style guide for the
JavaScript documentation simply because it's not needed. We don't really
have any for Python either (although there might be a lot of
python-specific stuff in what we have). We just have guidelines for
documenting code. Any code. JavaScript, Python, even templates.

Plus, the documentation generator that we are using already, Sphinx,
supports JavaScript perfectly fine, so I see no reason to add another tool.

I agree it would be nice to actually start using it for JavaScript code
too, though.

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] JavaScript docs?

[Radomir Dopieralski]

On 02/05/2015 10:27 AM, Anton Zemlyanov wrote:
 JSDoc (ngdoc) is good thing. It allows to describe files, functions and
 it's parameters, constructors, classes in case of ES6.

As does Sphinx.

 The problem is it tends to diverge with reality. The code is being fixed
 and evolved, but comments are often not updated (who want to do much
 more work)? And JSDoc generated documentation loses the connection to
 reality. 

That is not a problem. We will just reject patches that do such changes
without updating the comments.

-- 
Radomir Dopieralski

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 22/01/15 11:27, Martin Geisler wrote:
 Maybe this is a dumb question, but if there already is a system package
 for, say, Angular, why is the XStatic packge needed then? Could the
 system package for Horizon not just point directly to where the Angular
 package has put its files?

Yes, that is *exactly* the change that we want to make now and that we
are discussing! Drop the whole XStatic thing, and have a file in Horizon
that configures all the paths. Then use Bower for downloading the files
in development environments, and system packages in production.

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 21/01/15 19:04, Matthew Farina wrote:
 Radomir, thanks for adding some clarity. I do have follow-on questions.
 
 In your example the packages are managed by xstatic. The proposal for
 horizon, as I understand it, is to move away from xstatic packages and
 instead use bower for development and system packages (for example,
 debian, rpm, and other packages) for production. Right now, we (the
 horizon community) is maintaining some of the xstatic packages. For many
 of these xstatic packages there is no corollary in a system package. Who
 will create and maintain the system packages for the JavaScript libraries?
 
 You noted that we get maintenance and updates for free. Since the
 system packages don't exist now and we don't know who will create or
 maintain them I'm not sure how to reconcile this.
 
 What am I missing?

All of the XStatic packages had to be packaged for the respective
distributions in order to package Horizon. That was a lot of work, but
it has been done my the packagers of the distributions. As far as I
understand, most of those XStatic packages are just dummies, pointing to
the actual system-wide JavaScript packages -- XStatic has such a
capability. So while we are indeed maintaining some of the XStatic
packages for our own convenience, the packages that contain actual code
in the distributions are maintained by those distributions' packagers.

-- 
Radomir Dopieralski

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 20/01/15 20:58, Matthew Farina wrote:
 Radomir, maybe you can help me better understand where this would go. I
 have a few questions.
 
 First, can you point me to a time when horizon used system packages
 successfully for JavaScript libraries? When I looked through the Debian
 and Ubuntu packages I couldn't find the libraries horizon is using. I'm
 curious to see this in action.

Any distribution (perhaps except Ubuntu, which is a little funny in that
regard) that has packaged the latest release of OpenStack, has those
libraries.
For instance, see
http://pkgs.fedoraproject.org/cgit/python-django-horizon.git/tree/python-django-horizon.spec#n129


 Front-end systems almost never use system packagers like this. Can you
 point me to applications like horizon that use system packages this way?
 If Horizon is going to go it virtually alone in this space, what will
 that mean for our level of work and ability to have updates?

Certainly. The XStatic system itself is lifted from MoinMoin wiki, for
example -- it was created to solve exactly this problem there, and is
used by a couple of other projects too.

As for our work and updates, using system-wide packages is an excellent
solution in this regard, as we get maintenance and updates for free. For
instance, if there is a security issue in one of the JavaScript
libraries, we don't need to patch Horizon -- the patch that is prepared
for that specific library and applied system-wide is sufficient.
-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 21/01/15 09:21, david.co...@oracle.com wrote:
 As for our work and updates, using system-wide packages is an excellent
 solution in this regard, as we get maintenance and updates for free. For
 instance, if there is a security issue in one of the JavaScript
 libraries, we don't need to patch Horizon -- the patch that is prepared
 for that specific library and applied system-wide is sufficient.
 
 But for distributions that package Horizon itself, don't they
 effectively need to patch Horizon? Namely, don't they need to install
 on their build systems fixed JavaScript distribution packages to
 address the security issue and then they need to rebuild Horizon itself
 even if there are no Horizon source code changes.
 
 From a Horizon end-user perspective who relies on the distribution's
 packages to get Horizon, they'll get the security fix but it seems
 distributors will still need to rebuild and deliver Horizon for every
 upstream JavaScript fix whether the files come from XStatic, Bower, or
 some other method.

No, why would they? They don't copy the static files into the Horizon's
package. That's the whole point, Horizon only has paths to them. The
files themselves are provided by the system-wide packages.

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 16/01/15 18:55, Matthew Farina wrote:
 Doug, there still is one open question. Distributing JavaScript
 libraries via system packages is unusual. Because of that, most of the
 JavaScript libraries used by horizon don't have existing packages. Who
 will create and maintain the packages for these JavaScript libraries for
 production? For example, most of the libraries aren't available as
 debian or ubuntu packages.

You are mistaken here. It's actually the other way around. Fedora and
Debian packagers used to do heroic work with previous releases to
unbundle the static files from Horizon and link to the system-wide
JavaScript libraries installed from packages, because their packaging
policies require that. The introduction of XStatic was supposed to
merely simplify and formalize that process, but now it turns out that it
is redundant, and we can cut a corner and save the packagers having to
create all those dummy XStatic shims.

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 14/01/15 23:05, david.co...@oracle.com wrote:
 I'm not particularly well-versed in the Horizon build process so
 perhaps I'm way off base. But given that a distribution's Horizon build
 package embeds various JavaScript libraries to be used by the browser,
 how those libraries are obtained during the build process is an
 interesting build detail. I thought the intent of the XStatic work was
 to provide Python wrappers around the various JavaScript libraries so
 they could be pulled down via pip. Since there's already a Python
 requirements.txt file for versioning information, that seemed to be a
 nice way of indicating which versions of the JavaScript libraries could
 be used by Horizon and Python was used all the way through the build.
 
 I realize that it takes work to build the XStatic packages but given
 the Python packages are basically wrappers, it seems creating those and
 uploading them to pypi could be automated by using Bower and setup.py
 but perhaps translating the metadata isn't straightforward.

This has actually been discussed. The problem is, that it adds
considerable complexity and some amount of work, while solves exactly
zero problems. You are just sweeping the node.js dependency under the
rug, because now, instead of being a direct dependency (for development)
of Horizon, it becomes a dependency (for building) of a package that
Horizon is dependent on. That doesn't solve anything, since all
distributions insist on building their own packages from sources, not
relying on pre-built binaries.

So what we settled on is a compromise. Horizon will not care *how* the
static files got there. Whether you put them there by hand, use Bower,
or install your distribution's packages -- Horizon doesn't care. And
neither do the tests written for Horizon, or the build scripts. They
just assume the files are where you configured them. Period. Let me
repeat that:

***You will not need Node.js or Bower to build and test Horizon.***

The packagers for different distributions will want to make packages for
the JavaScript and CSS libraries on their own anyways, and they will
want to test Horizon with those -- not with what Bower installs.

The only place where you will need Bower (but only for convenience, as
you may as well install everything manually or with your own scripts) is
the development machine. For convenience, we will keep the list of
dependencies in a format compatible with Bower -- but it's a simple JSON
file that can be read by any other tool or script, written in any language.

I hope that clears it.

-- 
Radomir Dopieralski

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 13/01/15 16:13, Drew Fisher wrote:
 
 
 On 1/13/15 7:59 AM, Jeremy Stanley wrote:
 On 2015-01-13 08:50:28 +0100 (+0100), Matthias Runge wrote:
 [...]
 But, as far as I understand, node.js will become a development
 requirement (and most probably a requirement for testing), but not for
 deployment.
 [...]

 A requirement for testing _is_ a requirement for deployment. If it's
 not tested, it's broken. If you're deploying software on a platform
 where it can't be tested then you're simply _hoping_ that it's not
 broken, and that is almost certainly a false hope.

 
 Exactly.  We have to test this code base extensively before we package
 it up for Solaris.  Under no circumstances do we just blindly repackage
 the releases and push them out to customers.  Node.js is a total
 incompatibility for Solaris.  If upstream moves to using Bower we'll be
 forced to look for alternatives at managing the JavaScript libraries.

I have some bad news for you. Horizon already uses node.js for running
jshint on its JavaScript files on the gate. Simply because there is no
other alternative. You are welcome to work on and propose a better solution.

-- 
Radomir Dopieralski


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 06/01/15 01:53, Richard Jones wrote:
 I think the only outstanding question is how developers and
 non-packagers populate the bower_components directory - that is, how is
 bower expected to be available for them?
 
 I think following the Storyboard approach is a good idea: isolate a
 known-working node/bower environment local to horizon which is managed
 by tox - so to invoke bower you run tox -e bower command. No worries
 about system installation or compatibility, and works in the gate.
 
 Horizon installation (whenever a pip install would be invoked) would
 then also have a tox -e bower install invocation.
 
 Storyboard[1] uses a thing called nodeenv[2] which is installed through
 pip / requirements.txt to control the node environment. It then has
 bower commands in tox.ini[3] (though I'd just have a single bower
 environment to implement the tox command I suggest above.
 
  
  Richard
 
 [1] https://wiki.openstack.org/wiki/StoryBoard
 [2] https://pypi.python.org/pypi/nodeenv
 [3] 
 https://git.openstack.org/cgit/openstack-infra/storyboard-webclient/tree/tox.ini
 

I created a blueprint for this.
https://blueprints.launchpad.net/horizon/+spec/static-file-bower
-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 06/01/15 18:39, Lin Hua Cheng wrote:
 Radomir,
 
 The current version of Angular were using in Horizon still does not have
 cookie and mock
 packages: 
 https://github.com/stackforge/xstatic-angular/tree/1.2.1.1/xstatic/pkg/angular/data
 
 We still need to do it the long way:
 1. Update the Angular version in global-requirements
 2. Wait till it gets merge and propagate to horizon requirements
 3. Remove references loading of mock and cookie packages in horizon and
 horizon requirement
 4. Remove mock and cookie from global-requirements.

That's strange, I thought that we use 1.2.16 already. Sorry for my mistake.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 06/01/15 01:39, Tripp, Travis S wrote:
 What Radomir proposes looks like it would greatly ease the process I am still 
 going through to get the latest angular available to Horizon for current 
 development.  At the time of writing this, I’m still trying to get the 
 updated library through.  I hit a rather difficult workflow:
 
 
   1.  Packaged the latest into Xstatic-Angular-1.3.7
   2.  Submitted patch which deprecated the separate older 
 xstatic-angular-cookies and xstatic-angular-mock packages
   3.  Reviewed and approved (after correcting an initial mis-repackaging)
   4.  Radomir released to Pypi
 
 This was pretty easy and not too hard. Not too much to complain about.
 
 However, now, to get Horizon to use it, I have to get that into global 
 requirements.  Since I’m deprecating old packages I got stuck in a sort of 
 ugly dependency path.  I couldn’t remove the cookies and mock libraries from 
 the global requirements patch that added the new 1.3.7 package because of 
 horizon still referencing the deprecated packages.  And, when I did it 
 anyway, the integration tests failed due to horizon being dependent on 
 something not in global requirements.  So, now, as far as I can tell we have 
 to jump through the following hoops:
 
 
   1.  Global requirements patch to add angular 1.3.7
  *   Verify check / recheck fun
  *   Reviewed and approved
  *   Gate check / recheck fun
   2.  Horizon patch to update to angular 1.3.7 and remove deprecated mock and 
 cookies packages
  *   Verify check / recheck fun
  *   Reviewed and approved
  *   Gate check / recheck fun
   3.  Global requirements patch to remove deprecated mock and cookies
  *   Verify check / recheck fun
  *   Reviewed and approved
  *   Gate check / recheck fun
 
 Don’t get me wrong, I really do think the gate is brilliant and am all for a 
 review / approval process, but this does seem excessive for a UI library that 
 should only be used by Horizon. Is there some other reason that this should 
 have to go through global requirements?

You can do it much easier, since the current version of Angular already
packages what is in the deprecated modules. So just:

1. Patch Horizon to remove the xstatic dependencies to the mock and
cookies packages.
2. Patch global-requirements to remove them, and add newer Angular.
3. Patch Horizon to use the newer Angular.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 05/01/15 00:35, Richard Jones wrote:
 On Mon Dec 22 2014 at 8:24:03 PM Radomir Dopieralski
 openst...@sheep.art.pl mailto:openst...@sheep.art.pl wrote:
 
 On 20/12/14 21:25, Richard Jones wrote:
  This is a good proposal, though I'm unclear on how the
  static_settings.py file is populated by a developer (as opposed to a
  packager, which you described).
 
 It's not, the developer version is included in the repository, and
 simply points to where Bower is configured to put the files.
 So just to be clear, as developers we:

 1. have a bower.json listing the bower component to use,
 2. use bower to fetch and install those to the bower_components
 directory at the top level of the Horizon repos checkout, and
 3. manually edit static_settings.py when we add a new bower component to
 bower.json so it knows the appropriate static files to load from that
 component.

 Is that correct?

 The above will increase the burden on those adding or upgrading bower
 components (they'll need to check the bower.json in the component for
 the appropriate static files to link in) but will make life easier for
 the re-packagers since they'll know which files they need to cater for
 in static_settings.py

Well, I expect you can tell Bower to put the files somewhere else than
in the root directory of the project -- a directory like ``bower_files``
or something (that directory is also added to ``.gitignore`` so that you
don't commit it by mistake). Then only that directory needs to be added
to the ``static_settings.py``. Of course, you still need to make all the
``script`` links in appropriate places with the right URLs, but you
would have to do that anyways.

Let's look at an example. Suppose you need to a new JavaScript library
called hipster.js. You add it to the ``bower.json`` file, and run
Bower. Bower downloads the right files and does whatever it is that it
does to them, and puts them in  ``bower_files/hipster-js``. Now you edit
Horizon's templates and add ``script src={{ STATIC_URL
}}/hipster-js/hipster.js`` to ``_scripts.html``. That's it for you.
Since your ``static_settings.py`` file already has a line:

  ('', os.path.join(BASE_DIR, '/bower_files')),

in it, it will just work.

Now, suppose that a packager wants to package this for, say, Debian. And
suppose that Debian has hipster.js packaged, except it was called
bro.js before, and they left the old name for compatibility reasons.
He will look at the change history to the ``bower.json`` and the
``_scripts.html`` files, take the ``static_settings.py`` file for his
distribution, and add a line:

  ('hipster-js/hipster.js', '/usr/lib/js_libraries/bro_js/bro.js'),


-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

On 20/12/14 21:25, Richard Jones wrote:
 This is a good proposal, though I'm unclear on how the
 static_settings.py file is populated by a developer (as opposed to a
 packager, which you described).

It's not, the developer version is included in the repository, and
simply points to where Bower is configured to put the files.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] static files handling, bower/

[Radomir Dopieralski]

Hello,

revisiting the package management for the Horizon's static files again,
I would like to propose a particular solution. Hopefully it will allow
us to both simplify the whole setup, and use the popular tools for the
job, without losing too much of benefits of our current process.

The changes we would need to make are as follows:

* get rid of XStatic entirely;
* add to the repository a configuration file for Bower, with all the
required bower packages listed and their versions specified;
* add to the repository a static_settings.py file, with a single
variable defined, STATICFILES_DIRS. That variable would be initialized
to a list of pairs mapping filesystem directories to URLs within the
/static tree. By default it would only have a single mapping, pointing
to where Bower installs all the stuff by default;
* add a line from static_settings import STATICFILES_DIRS to the
settings.py file;
* add jobs both to run_tests.sh and any gate scripts, that would run Bower;
* add a check on the gate that makes sure that all direct and indirect
dependencies of all required Bower packages are listed in its
configuration files (pretty much what we have for requirements.txt now);

That's all. Now, how that would be used.

1. The developers will just use Bower the way they would normally use
it, being able to install and test any of the libraries in any versions
they like. The only additional thing is that they would need to add any
additional libraries or changed versions to the Bower configuration file
before they push their patch for review and merge.

2. The packagers can read the list of all required packages from the
Bower configuration file, and make sure they have all the required
libraries packages in the required versions.

Next, they replace the static_settings.py file with one they have
prepared manually or automatically. The file lists the locations of all
the library directories, and, in the case when the directory structure
differs from what Bower provides, even mappings between subdirectories
and individual files.

3. Security patches need to go into the Bower packages directly, which
is good for the whole community.

4. If we aver need a library that is not packaged for Bower, we will
package it just as we had with the XStatic packages, only for Bower,
which has much larger user base and more chance of other projects also
using that package and helping with its testing.

What do you think? Do you see any disastrous problems with this system?
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon]

[Radomir Dopieralski]

On 25/11/14 00:09, David Lyle wrote:
 I am pleased to nominate Thai Tran and Cindy Lu to horizon-core. 
[...]

Thai Tran +1
Cindy Lu +1

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 21/11/14 06:12, Thomas Goirand wrote:

 Let's say there's python-xstatic-foo, and libjs-foo in Debian. If the
 directory structure of libjs-foo is very different from xstatic-foo, I
 can address that issue with symlinks within the xstatic package. Just
 changing the BASE_DIR may not be enough, as libjs-foo may have files
 organized in a very different way than in the upstream package for foo.

You can do it without xstatic even more easily, without having to muck
around with symlinks. Django has this STATICFILES_DIRS setting variable,
that you can set, that lets you specify on what URL each static
directory should be available. For instance, I use it currently to work
around the issue with jquery-ui changing directory structure between
versions:

if xstatic.main.XStatic(xstatic.pkg.jquery_ui).version.startswith('1.10.'):
# The 1.10.x versions already contain the 'ui' directory.
STATICFILES_DIRS.append(
('horizon/lib/jquery-ui',
 xstatic.main.XStatic(xstatic.pkg.jquery_ui).base_dir))
else:
# Newer versions dropped the directory, add it to keep the path the
same.
STATICFILES_DIRS.append(
('horizon/lib/jquery-ui/ui',
 xstatic.main.XStatic(xstatic.pkg.jquery_ui).base_dir))

But in your Debian package, you can completely drop xstatic and just use
absolute paths to the filesystem explicitly.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 18/11/14 00:59, Richard Jones wrote:
 On 17 November 2014 21:54, Radomir Dopieralski openst...@sheep.art.pl
 mailto:openst...@sheep.art.pl wrote:

 - Bower in the development environment,
 - Bower configuration file in two copies, one for global-requirements,
 and one for the Horizon's local requirements. Plus a gate job that makes
 sure no new library or version gets included in the Horizon's before
 getting into the global-requirements,
 
 Could you perhaps elaborate on this? How do you see the workflow working
 here?

Basically we would have an additional file in the global-requirements
directory, for listing the JavaScript dependencies. Then we would have a
check on the Horizon's gate that would check Horizon's Bower
configuration against that global-requirements file.

This way we keep the same process for JavaScript libraries as we have
for Python libraries: first you submit a patch to the
global-requirements and have the dependency accepted by the infra team
and packagers (checked against licenses, version conflicts, etc.), and
then you add it to Horizon's dependencies. Of course you can submit both
patches at once, just the Horizon one will fail the gate until the
global-requirements one gets merged.

 Given that Horizon already integrates with xstatic, it would be messy
 (and potentially confusing) to include something so it *also* integrated
 with bower. I was envisaging us creating a tool which generates xstatic
 packages from bower packages. I'm not the first to think along these
 lines 
 http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

If we use Bower, we don't need to use Xstatic. It would be pure
overhead. Bower already takes care of tracking releases and versions,
and of bundling the files. All we need is a simple line in the
settings.py telling Django where it puts all the files -- we don't
really need Xstatic just for that. The packagers can then take those
Bower packages and turn them into system packages, and just add/change
the paths in settings.py to where they put the files. All in one place.

 I will be looking into creating such a tool today.

The problem is not the work that has to be done for the initial creation
of the package. That is one-time effort and as you show, it can be
easily automated. The problem is the effort and resources needed to
maintain that package. Someone (the author of the package?) has to check
for security vulnerabilities, critical bugs, packaging issues, changing
licenses, etc. and patch/update the packages accordingly. Also, the more
layers of code you have, them more likely you are to have bugs in the.
Again, I see no reason to duplicate the effort if the Bower packagers
are doing that work for us already (they are, are they?).

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 18/11/14 12:54, Matthias Runge wrote:
 On Tue, Nov 18, 2014 at 09:36:00PM +1100, Richard Jones wrote:
 I guess I got the message that turning bower packages into system packages
 was something that the Linux packagers were not keen on. Did I get the
 message wrong there? If so, *and* we can get the bower stuff through #infra
 and global-requirements then yes, we should totally try to avoid adding the
 xstatic layer :)
 
 For me, it's more work to turn packages into bower, or to translate
 bower packages to system packages.
 
 But that is purely because we don't have bower packaged currently in Fedora.
 I would vote for the cleaner way (whatever that is).
 
 XStatic is obviously not the cleanest way, but a good compromise in most
 situations.

The way I thought about it, we would simply replace the Bower packages
with the corresponding system packages, by just changing the path in the
settings.py file.

Maybe an example would help illustrate it.
Currently we have something like this:

STATICFILES_DIRS = [
('horizon/lib/angular',
   xstatic.main.XStatic(xstatic.pkg.angular).base_dir),
('horizon/lib/angular',
   xstatic.main.XStatic(xstatic.pkg.angular_cookies).base_dir),
('horizon/lib/angular',
   xstatic.main.XStatic(xstatic.pkg.angular_mock).base_dir),
('horizon/lib/bootstrap_datepicker',
   xstatic.main.XStatic(xstatic.pkg.bootstrap_datepicker).base_dir),
('bootstrap',
   xstatic.main.XStatic(xstatic.pkg.bootstrap_scss).base_dir),
...
]

We would replace that with:

STATICFILES_DIRS = [
('horizon/lib/angular',
   os.path.join(BASE_DIR, 'bower_modules/angular'),
...
]

or wherever bower puts the files. Now, the packagers would replace those
lines with:

STATICFILES_DIRS = [
('horizon/lib/angular',
   '/usr/lib/javascript/angular'),
...
]

or whatever the packages in their distribution use. It's less work than
having to make a whole xstatic package just to put that single path in
there. The horizon system package would already depend on all the
javascript library packages, so we are sure the files are there.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 17/11/14 09:53, Martin Geisler wrote:

[...]

 As Richard said, npm and bower are not competitors. You use npm to
 install bower, and you use bower to download Angular, jQuery, Bootstrap
 and other static files. These are the static files that you will want to
 include when you finally deploy the web app to your server.
 
 Before using Bower, people would simply download Angular from the
 projects homepage and check it into version control. Bower is not doing
 much, but using it avoids this bad practice.
 
 There is often a kind of compilation step between bower downloading a
 dependency and the deployment on the webserver: minification and
 compression of the JavaScript and CSS. Concatenating and minifying the
 files serve to reduce the number of HTTP requests -- which can make an
 app much faster.
 
 Finally, you use Grunt/Gulp to execute other tools during development.
 These tools could be a local web server, it could be running the unit
 tests. Grunt is only a convenience tool here -- think of it as a kind of
 Makefile that tells you how to lunch various tasks.

Thank you for your explanations.

The way I see it, we would need:
- Bower in the development environment,
- Grunt both in the development environment and packaged (to run tests,
etc.),
- Bower configuration file in two copies, one for global-requirements,
and one for the Horizon's local requirements. Plus a gate job that makes
sure no new library or version gets included in the Horizon's before
getting into the global-requirements,
- A tool, probably a script, that would help packaging the Bower
packages into DEB/RPM packages. I suspect the Debian/Fedora packagers
already have a semi-automatic solution for that.
- A script that would generate a file with all the paths to those
packaged libraries, that would get included in Horizon's settings.py

What do you think?
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 15/11/14 03:21, Richard Jones wrote:
 On 15 November 2014 00:58, Radomir Dopieralski openst...@sheep.art.pl wrote:

[...]

 4. additions and upgrades of libraries moderated by the packagers,
 
 Is there already some mechanism for handling the creation and management
 of xstatic packages and how they interact with linux packagers? Sorry if
 this is a noob question.

Anybody can at any time create any Xstatic package and put it on PyPi.
To put it in StackForge, it has to be approved by the infra team, but
they usually don't make problems (also, you don't technically need to
keep the source code on StackForge). To put it in the global
requirements.txt file, it has to be approved by the people who review
that repository, and that includes the packagers. To put it in the
Horizon's requirements.txt, it has to be approved by the Horizon core
reviewers.

I imagine we can have a similar setup for JavaScript dependencies,
possibly a bower configuration file, that would be handled in a similar way.

[...]
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 14/11/14 13:02, Richard Jones wrote:
 On 14 November 2014 18:51, Radomir Dopieralski openst...@sheep.art.pl
 mailto:openst...@sheep.art.pl wrote:
 On 13/11/14 23:30, Martin Geisler wrote:
[...]

  Maybe a difference is that you don't (yet) install a web application
  like you install a system application. Instead you *deploy* it: you
  unpack files on a webserver, you configure permissions, you setup cache
  rules, you configure a database, etc.

[...]

 In order to do that, they can't just take a tarball and drop it in a
 directory. They always produce their own builds, to make sure it's the
 same thing that the source code specifies. They sometimes have to
 rearrange configuration files, to make them fit the standards in their
 system. They provide sane configuration defaults. They track the
 security reports about all the installed components, and apply fixes,
 often before the security issue is even announced.

[...]


 I think that it boils down to whether it'is possible that
 distributions:

 1. package the node-based tools (grunt, karma, protractor, ...) as
 installable programs, and
 2. xstatic-package the bower-based packages that we use (probably a
 couple of dozen at least).

 We might even be able to get away without using grunt, though an
 alternative to its LiveReload facility (and one that doesn't then also
 depend on another node program like django-livereload does) would be
 required. I believe tox and django's runserver (and other manage
 commands) could suffice to replace the other functionality typically
 provided by grunt.

We don't really need Xstatic for that. The packagers can as well package
the bower-based packages directly. We can use anything, really,
as long as we follow a process that makes sure that Horizon can be
packaged into the different distributions. That is, we need:

1. All dependencies explicit (with tests failing if a dependency is
   missing),
2. explicit version ranges,
3. no multiple versions of the same library,
4. additions and upgrades of libraries moderated by the packagers,
5. ability to replace the development environment with packaged
   libraries from the system,
6. ability to build and test our software with the tools that can be
   used by all the distributions.

As I said, this is more of a process thing than a tool thing -- I
believe any tool can be used to follow this process, more or less
automatically.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 11/11/14 08:02, Richard Jones wrote:

[...]

 There were some discussions around tooling. We're using xstatic to
 manage 3rd party components, but there's a lot missing from that
 environment. I hesitate to add supporting xstatic components on to the
 already large pile of work we have to do, so would recommend we switch
 to managing those components with bower instead. For reference the list
 of 3rd party components I used in angboard* (which is really only a
 teensy fraction of the total application we'd end up with, so this
 components list is probably reduced):

[...]

 Just looking at PyPI, it looks like only a few of those are in xstatic,
 and those are out of date.

There is a very good reason why we only have a few external JavaScript
libraries, and why they are in those versions.

You see, we are not developing Horizon for our own enjoyment, or to
install it at our own webserver and be done with it. What we write has
to be then packaged for different Linux distributions by the packagers.
Those packagers have very little wiggle room with respect to how they
can package it all, and what they can include.

In particular, libraries should get packaged separately, so that they
can upgrade them and apply security patches and so on. Before we used
xstatic, they have to go through the sources of Horizon file by file,
and replace all of our bundled files with symlinks to what is provided
in their distribution. Obviously that was laborious and introduced bugs
when the versions of libraries didn't match.

So now we have the xstatic system. That means, that the libraries are
explicitly listed, with their minimum and maximum version numbers, and
it's easy to make a dummy xstatic package that just points at some
other location of the static files. This simplifies the work of the
packagers.

But the real advantage of using the xstatic packages is that in order to
add them to Horizon, you need to add them to the global-requirements
list, which is being watched and approved by the packagers themselves.
That means, that when you try to introduce a new library, or a version
of an old library, that is for some reason problematic for any of the
distributions (due to licensing issues, due to them needing to remain at
an older version, etc.), they get to veto it and you have a chance of
resolving the problem early, not dropping it at the last moment on the
packagers.

Going back to the versions of the xstatic packages that we use, they are
so old for a reason. Those are the newest versions that are available
with reasonable effort in the distributions for which we make Horizon.

If you want to replace this system with anything else, please keep in
contact with the packagers to make sure that the resulting process makes
sense and is acceptable for them.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 13/11/14 08:23, Matthias Runge wrote:

[...]

 Since we don't require node.js on the server (yet), but only for
 the development process: did anyone look at node's competitors? Like
 CommonJS, Rhino, or SpiderMonkey?

When we were struggling with adding jslint to our CI, we did try a
number of different alternatives to node.js, like Rhino, SpiderMonkey,
V8, phantomjs, etc.

The conclusion was that even tools that advertised themselves as working
on Rhino dropped their support for it several years ago, and just didn't
update the documentation. Node seems to be the only thing that works
without having to modify the code of those tools.

Of course things might have changed since, or we may have someone with
better JavaScript hacking skills who would manage to make it work. But
last year we failed.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 13/11/14 01:32, Richard Jones wrote:
[...]

 We're currently using xstatic and that works with Linux packaging
 because it was designed to cope with being a global installation. The
 current Horizon codebase has a django-xstatic plugin which further makes
 dealing with xstatic components nicer - for example it handles path
 management and static file compilation (JS minification and
 concatenation, for example). That's really nice, but poses some problems:
 
 - we would need to xstatic-ify (and deb/rpm-ify) all those components

Yes. They will need to be deb/rpm/arch/slack/whatever-ified anyways,
because that's how the Linux distributions that are going to ship them work.

 - we could run into global version conflict issues if we run more than
 one service on a system - is this likely to be an issue in practise though?

Yes, this is an issue in practice, and that's why the packagers have a
say in what libraries and in what versions you are adding to the
global-requirements. We have to use versions that are the least problematic.

 - as far as I'm aware, the xstatic JS minification is not angular-aware,
 and will break angular code that has not been written to be
 dumb-minifier-aware (the angular minifier ngMin is written in node and
 knows how to do things more correctly); adding dumb-minifier-awareness
 to angular code makes it ugly and more error-prone :/

You can use any minifier with the django-compress plugin that Horizon
uses (django-xstatic has nothing to do with it). You just define the
command (or a filter written in Python) to use for every mime type.

But I assume that ngMin is written in the Node.js language (which is
superficially similar to JavaScript) and therefore if we used it, you
would have to convince your fellow system administrators to install
node.js on their production servers. Violence may result.

[...]
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Separate horizon and openstack_dashboard

[Radomir Dopieralski]

On 30/10/14 13:13, Matthias Runge wrote:
 I tried that in [3], [4]. I renamed openstack_dashboard to
 openstack_horizon, rather than horizon to be sure, I really catched all
 imports etc., and to make sure, it's clear, what component is meant.
 During this process, the name horizon is a bit ambiguous.

Thank you for doing this, it helps a lot!

[snip]

 So, how do we proceed from here?
 - how do we block the gate

I have no idea, in the past we just -2-ed all patches temporarily?

 - how to create a new repo
 - how to set up ci for the new project?

That shouldn't be too complicated. Basically you send a patch to
https://github.com/openstack-infra/project-config and have it accepted
by the infra team. I suppose we will copy most of the current horizon
setup. It will help to have someone from infra helping us on that.

 - how to integrate new horizon_lib and horizon (or openstack-horizon) to
 devstack

I suppose we have to reach out to the devstack people for that.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

[Radomir Dopieralski]

On 13/11/14 23:30, Martin Geisler wrote:

[...]

 While I agree that it's chaotic, I also think you make the problem worse
 than it really is. First, remember that the user who installs Horizon
 won't need to use the JavaScript based *developer* tools such as npm,
 bower, etc.
 
 That is, I think Horizon developers will use these tools to produce a
 release -- a tarball -- and that tarball will be something you unpack on
 your webserver and then you're done. I base this on what I've seen in
 the project I've been working. The release tarball you download here
 don't mention npm, bower, or any of the other tools:
 
   https://github.com/zerovm/swift-browser/releases
 
 The tools were used to produce the tarball and were used to test it, but
 they're not part of the released product. Somewhat similar to how GCC
 isn't included in the tarball if you download a pre-compiled binary.

[...]

 Maybe a difference is that you don't (yet) install a web application
 like you install a system application. Instead you *deploy* it: you
 unpack files on a webserver, you configure permissions, you setup cache
 rules, you configure a database, etc.

[...]

I think I see where the misunderstanding is in this whole discussion. It
seems it revolves around the purpose and role of the distribution.

From the naive point of view, the role of a Linux distribution is to
just collect all the software from respective upstream developers and
put it in a single repository, so that it can be easily installed by the
users. That's not the case.

The role of a distribution is to provide a working ecosystem of
software, that is:
a) installed and configured in consistent way,
b) tested to work with the specific versions that it ships with,
c) audited for security,
d) maintained, including security patches,
e) documented, including external tutorials and the like,
f) supported, either by the community or by the companies that provide
support,
g) free of licensing issues and legal risks as much as possible,
h) managed with the common system management tools.

In order to do that, they can't just take a tarball and drop it in a
directory. They always produce their own builds, to make sure it's the
same thing that the source code specifies. They sometimes have to
rearrange configuration files, to make them fit the standards in their
system. They provide sane configuration defaults. They track the
security reports about all the installed components, and apply fixes,
often before the security issue is even announced.

Basically, a distribution adds a whole bunch of additional guarantees
for the software they ship. Those are often long-term guarantees, as
they will be supporting our software long after we have forgotten about
it already.

You say that web development doesn't work like that. That may be true,
but that's mostly because if you develop a new web application in-house,
and deploy it on your server, you don't really have such large legal
risk, configuration complexity or support problem -- you just have to
care about that single application, because the packagers from the
distribution that you are using are taking care about all the rest of
software on your server.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon]Blueprint- showing a small message to the user for browser incompatibility

[Radomir Dopieralski]

On 14/10/14 18:30, Aggarwal, Nikunj wrote:

 Instead Horizon guys came to an conclusion that we will identify the
 browser type and version to deal with legacy browsers like older IE
 or firefox or any other browser and for other major features we can
 use feature detection with Modernizr.

I remember that discussion differently, and I'm not so sure there was a
definite conclusion. We definitely should not use a white list for this.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Problem with compressing scss files

[Radomir Dopieralski]

On 09/28/2014 09:41 AM, Thomas Goirand wrote:
 On 09/28/2014 03:35 PM, Thomas Goirand wrote:
 After a long investigation, I have found out that, in python-pyscss,
 there's the following code in scss/expression.py:

 return String(
 six.u(%s(%s) % (func_name, six.u(, .join(rendered_args,
 quotes=None)

 If I remove the first six.u(), and the code becomes like this:

 return String(
 %s(%s) % (func_name, six.u(, .join(rendered_args))),
 quotes=None)

 then everything works. Though this comes from a Debian specific patch
 for which I added the six.u() calls, to make it work in Python 3.2 in
 Wheezy. The original code is in fact:

 return String(
 u%s(%s) % (func_name, u, .join(rendered_args)),
 quotes=None)

 So, could anyone help me fixing this? What's the way to make it always
 work? I wouldn't like to just drop Python 3.x support because of this... :(
 
 Ops, silly me. The parenthesis aren't correct. Fixing it made it all
 work. Sorry for the noise, issue closed!

By the way, did you consider sending that python 3 patch upstream to the
python-pyscss guys, so that you don't have to apply it manually every
time? They are quite responsive.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] Deprecating exceptions

[Radomir Dopieralski]

Horizon's tests were recently broken by a change in the Ceilometer
client that removed a deprecated exception. The exception was deprecated
for a while already, but as it often is, nobody did the
work of removing all references to it from Horizon before it was too
late. Sure, in theory we should all be reading the release notes of
all versions of all dependencies and acting upon things like this.
In practice, if there is no warning generated in the unit tests,
nobody is going to do anything about it.

So I sat down and started thinking about how to best generate a warning
when someone is trying to catch a deprecated exception. I came up with
this code:

http://paste.openstack.org/show/114170/

It's not pretty -- it is based on the fact that the `except` statement
has to do a subclass check on the exceptions it is catching. It requires
a metaclass and a class decorator to work, and it uses a global
variable. I'm sure it would be possible to do it in a little bit cleaner
way. But at least it gives us the warning (sure, only if an exception is
actually being thrown, but that's test coverage problem).

I propose to do exception deprecating in this way in the future.
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] PYTHONDONTWRITEBYTECODE=true in tox.ini

[Radomir Dopieralski]

On 12/09/14 17:11, Doug Hellmann wrote:

 I also use git-hooks with a post-checkout script to remove pyc files any time 
 I change between branches, which is especially helpful if the different 
 branches have code being moved around:
 
 git-hooks: https://github.com/icefox/git-hooks
 
 The script:
 
 $ cat ~/.git_hooks/post-checkout/remove_pyc
 #!/bin/sh
 echo Removing pyc files from `pwd`
 find . -name '*.pyc' | xargs rm -f
 exit 0

Good thing that python modules can't have spaces in their names! But for
the future, find has a -delete parameter that won't break horribly on
strange filenames.

find . -name '*.pyc' -delete

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Some thoughts about Horizon's test suite

[Radomir Dopieralski]

On 29/08/14 04:22, Richard Jones wrote:


 Very recently I attempted to fix a simple bug in which a Panel was being
 displayed when it shouldn't have been. The resultant 5-line fix ended up
 breaking 498 of the 1048 unit tests in the suite. I estimated that it
 would take about a week's effort to address all the failing tests. For
 more information see
 http://mechanicalcat.net/richard/log/Python/When_testing_goes_bad

Having read that, I can't help but comment that maybe, just maybe,
making an API call on each an every request to Horizon is not such a
great idea after all, and should be very well thought out, as it is
costly. In particular, it should be investigated if the call could be
made only on some requests? That would have the side effect of breaking
much fewer tests.

But I agree that mox is horrible in that it effectively freezes the
implementation details of the tested unit, instead of testing its behavior.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [glance] python namespaces considered harmful to development, lets not introduce more of them

[Radomir Dopieralski]

On 27/08/14 16:31, Sean Dague wrote:

[snip]

 In python 2.7 (using pip) namespaces are a bolt on because of the way
 importing modules works. And depending on how you install things in a
 namespace will overwrite the base __init__.py for the top level part of
 the namespace in such a way that you can't get access to the submodules.
 
 It's well known, and every conversation with dstuft that I've had in the
 past was don't use namespaces.

I think this is actually a solved problem. You just need a single line
in your __init__.py files:

https://bitbucket.org/thomaswaldmann/xstatic-jquery/src/tip/xstatic/__init__.py

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [glance] python namespaces considered harmful to development, lets not introduce more of them

[Radomir Dopieralski]

On 28/08/14 12:41, Radomir Dopieralski wrote:
 On 27/08/14 16:31, Sean Dague wrote:
 
 [snip]
 
 In python 2.7 (using pip) namespaces are a bolt on because of the way
 importing modules works. And depending on how you install things in a
 namespace will overwrite the base __init__.py for the top level part of
 the namespace in such a way that you can't get access to the submodules.

 It's well known, and every conversation with dstuft that I've had in the
 past was don't use namespaces.
 
 I think this is actually a solved problem. You just need a single line
 in your __init__.py files:
 
 https://bitbucket.org/thomaswaldmann/xstatic-jquery/src/tip/xstatic/__init__.py
 
More on that at
http://pythonhosted.org/setuptools/setuptools.html?namespace-packages#namespace-packages

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Criteria for giving a -1 in a review

[Radomir Dopieralski]

On 21/08/14 18:05, Matthew Booth wrote:

[snip]

 This seems to mean different things to different people. There's a list
 here which contains some criteria for new commits:

[snip]


 Any more of these?

There is also https://wiki.openstack.org/wiki/CodeReviewGuidelines

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] Customization use cases

[Radomir Dopieralski]

Hello,

I started a page on the wiki to collect the stories about people
customizing Horizon here:
https://wiki.openstack.org/wiki/Horizon/Customization

I hope that it will help us improve Horizon's customization support in
the areas where people actually care about it, and make it more useful.

If you are customizing or extending Horizon, please take a moment and
write what you are doing exactly and what you would like to do, if it's
not possible at the moment. Please do it under a new heading, so that we
have some order in there.

Thank you,
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] request to review bug 1301359

[Radomir Dopieralski]

On 08/07/14 08:29, Harshada Kakad wrote:

 HI All,
 Could someone please, review the bug 
[...]

Hello Harshada,

please don't send such e-mails or ask in the IRC, submitting the patch
is enough to get reviewers to read it, eventually. If we sent such an
e-mail for every patch, this mailing list would become useless quickly.

Please note, however, that the patch queue for Horizon is quite long and
we only have so many people doing reviews, so it can take a long time.
If you would like to speed up the whole process, please help us by
reviewing some patches yourself. That will help us review faster and get
to your patch sooner.

Thank you,
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Nominations to Horizon Core

[Radomir Dopieralski]

On 20/06/14 23:17, Lyle, David wrote:
 I would like to nominate Zhenguo Niu and Ana Krivokapic to Horizon core.
 
 Zhenguo has been a prolific reviewer for the past two releases providing
 high quality reviews. And providing a significant number of patches over
 the past three releases.
 
 Ana has been a significant reviewer in the Icehouse and Juno release
 cycles. She has also contributed several patches in this timeframe to both
 Horizon and tuskar-ui.
 
 Please feel free to respond in public or private your support or any
 concerns.

Ana +1
Zhenguo +1

Thank you for your great work guys!

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] Configuration files RFC

[Radomir Dopieralski]

Hello,

I did some planning and thinking around the subject of Horizon's
configuration files. I summarized it all at:

https://review.openstack.org/#/c/100521/8/horizon-config-rfc.rst

Please feel free to comment. Any feedback appreciated.
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Quick Survey: Horizon Mid-Cycle Meetup

[Radomir Dopieralski]

On 20/06/14 13:56, Jaromir Coufal wrote:
 On 2014/19/06 09:58, Matthias Runge wrote:
 On Wed, Jun 18, 2014 at 10:55:59AM +0200, Jaromir Coufal wrote:
 My quick questions are:
 * Who would be interested (and able) to get to the meeting?
 * What topics do we want to discuss?

 https://etherpad.openstack.org/p/horizon-juno-meetup

 Thanks for bringing this up!

 Do we really have items to discuss, where it needs a meeting in person?

 Matthias
 
 I am not sure TBH, that's why I added also the Topic section to figure
 out if there is something what needs to be discussed. Though I don't see
 much interest yet.

Apart from the split, I also work on configuration files rework, which
could benefit from discussion, but i think it's better done here or on
the wiki/etherpad, as that leaves tangible traces. I will post a
detailed e-mail in a few days. Other than that, I don't see a compelling
reason to organize it.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Name proposals

[Radomir Dopieralski]

On 06/10/2014 09:18 PM, Radomir Dopieralski wrote:

The name poll is now officially over, and the winner is:

horizon_lib

You can view the results here:
http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_ea99af9511f3f255

I think we won't need to check for trademark issues with this name, so
we can just proceed with the split.

Thank you everyone for your input!
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Name proposals

[Radomir Dopieralski]

On 06/11/2014 01:16 PM, Jaromir Coufal wrote:

Try voting from outside of the VPN. The anonymous votes are one per IP
address.

 Hm, strange.
 
 I have contributed in Icehouse but didn't get the e-mail fro voting. I
 wanted to vote from the link which you provided but it didn't work for
 me - you already voted from given key.
 
 Can anybody help?
 
 Thanks
 -- Jarda
 
 On 2014/10/06 21:18, Radomir Dopieralski wrote:
 Hello everyone.

 We have collected a fine number of name proposals for the library part
 of Horizon, and now it is time to vote for them. I have set up a poll on
 CIVS, and if you contributed to Horizon within the last year, you should
 receive an e-mail with the link that lets you vote.

 If you didn't receive an e-mail, but you would like to vote anyways,
 you can do so by visiting the following URL:

 http://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_ea99af9511f3f255akey=e4c064fca36f8d26


 Please rank the names from the best to the worst. The names that were
 obvious conflicts or intended for the dashboard part were removed from
 the list. Once we get the results, we will consult with the OpenStack
 Foundation and select the first valid name with the highest ranking.

 The poll will end at the next Horizon team meeting, Tuesday, June 17, at
 16:00 UTC.

 Thank you,

 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Name proposals

[Radomir Dopieralski]

Hello everyone.

We have collected a fine number of name proposals for the library part
of Horizon, and now it is time to vote for them. I have set up a poll on
CIVS, and if you contributed to Horizon within the last year, you should
receive an e-mail with the link that lets you vote.

If you didn't receive an e-mail, but you would like to vote anyways,
you can do so by visiting the following URL:

http://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_ea99af9511f3f255akey=e4c064fca36f8d26

Please rank the names from the best to the worst. The names that were
obvious conflicts or intended for the dashboard part were removed from
the list. Once we get the results, we will consult with the OpenStack
Foundation and select the first valid name with the highest ranking.

The poll will end at the next Horizon team meeting, Tuesday, June 17, at
16:00 UTC.

Thank you,
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon][infra] Plan for the splitting of Horizon into two repositories

[Radomir Dopieralski]

On 06/05/2014 10:59 AM, Mark McLoughlin wrote:

 If I'm late to the party and the only one that this is news to, that is
 fine. Sixteen additional repos seems like a lot of additional reviews
 will be needed.
 
 One slightly odd thing about this is that these repos are managed by
 horizon-core, so presumably part of the Horizon program, but yet the
 repos are under the stackforge/ namespace.

What would you propose instead?
Keeping them in repositories external to OpenStack, on github or
bitbucket sounds wrong.
Getting them under openstack/ doesn't sound good either, as the
projects they are packaging are not related to OpenStack.

Have them be managed by someone else? Who?

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Name proposals

[Radomir Dopieralski]

On 06/03/2014 06:44 PM, Radomir Dopieralski wrote:
 We decided that we need to pick the name for the splitting of Horizon
 properly. From now up to the next meeting on June 10 we will be
 collecting name proposals at:
 
 https://etherpad.openstack.org/p/horizon-name-proposals
 
 After that, until next meeting on June 17, we will be voting for
 the proposed names. In case the most popular name is impossible to use
 (due to trademark issues), we will use the next most popular. In case of
 a tie, we will pick randomly.

Just a quick remark. I allowed myself to clean the etherpad up a little,
putting all the proposed names on a single list, so that we can vote on
them later easily.

I would like to remind all of you, that the plan for the split itself is
posted on a separate etherpad, and that if anybody has any suggestions
for any additions or changes to the plan, I'd love to see them there:

https://etherpad.openstack.org/p/horizon-split-plan

That includes the suggestions for renaming the other part, and the
changes in the plan and in the later organization that would stem from
such a change.

Thank you,

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] mocking policy

[Radomir Dopieralski]

Hello,

I'd like to start a discussion about the use of mocking libraries in
Horizon's tests, in particular, mox and mock.

As you may know, Mox is the library that has been used so far, and we
have a lot of tests written using it. It is based on a similar Java
library and does very strict checking, although its error reporting may
leave something more to be desired.

Mock is a more pythonic library, insluded in the stdlib of recent Python
versions, but also available as a separate library for older pythons. It
has a much more relaxed approach, allowing you to only test the things
that you actually care about and to write tests that don't have to be
rewritten after each and every refactoring.

Some OpenStack projects, such as Nova, seem to have adopted an approach
that favors Mock in newly written tests, but allows use of Mox for older
tests, or when it's more suitable for the job.

In Horizon we only use Mox, and Mock is not even in requirements.txt. I
would like to propose to add Mock to requirements.txt and start using it
in new tests where it makes more sense than Mox -- in particular, when
we are writing unit tests only testing small part of the code.

Thoughts?
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon][infra] Plan for the splitting of Horizon into two repositories

[Radomir Dopieralski]

On 05/31/2014 11:13 PM, Jeremy Stanley wrote:
 On 2014-05-29 20:55:01 + (+), Lyle, David wrote:
 [...]
 There are several more xstatic packages that horizon will pull in that are
 maintained outside openstack. The packages added are only those that did
 not have existing xstatic packages. These packages will be updated very
 sparingly, only when updating say bootstrap or jquery versions.
 [...]
 
 I'll admit that my Web development expertise is probably almost 20
 years stale at this point, so forgive me if this is a silly
 question: what is the reasoning against working with the upstreams
 who do not yet distribute needed Javascript library packages to help
 them participate in the distribution channels you need?

There is nothing stopping us from doing that. On the other hand,
I don't expect much success with that. Those are JavaScript and/or
style/resources libraries, and the authors usually don't know or
care about python packaging. Some of those libraries don't even have
proper releases or versioning! We can reach out and ask them to
include the packages in their releases (where they have them), but
we need the packages now -- we are already using those libraries and
we need to clean up how we bundle them.

 This strikes
 me as similar to forking a Python library which doesn't publish to
 PyPI, just so you can publish it to PyPI.

There is no fork, as we are not modifying the source code.

 When some of these
 dependencies begin to publish xstatic packages themselves, do the
 equivalent repositories in Gerrit get decommissioned at that point?

Yes, we will hand over the keys to the pypi entries, and get rid of the
repositories on our side.
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] Name proposals

[Radomir Dopieralski]

We decided that we need to pick the name for the splitting of Horizon
properly. From now up to the next meeting on June 10 we will be
collecting name proposals at:

https://etherpad.openstack.org/p/horizon-name-proposals

After that, until next meeting on June 17, we will be voting for
the proposed names. In case the most popular name is impossible to use
(due to trademark issues), we will use the next most popular. In case of
a tie, we will pick randomly.
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Use of AngularJS

[Radomir Dopieralski]

On 06/02/2014 05:13 PM, Adam Nelson wrote:
 I think that you would use the PyPI version anyway:
 
 https://pypi.python.org/pypi/django-angular/0.7.2
 
 That's how most of the other Python dependencies work, even in the
 distribution packages.

That is not true. As all components of OpenStack, Horizon has to be
packaged at the end of the cycle, with all of its dependencies.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon][infra] Plan for the splitting of Horizon into two repositories

[Radomir Dopieralski]

On 05/29/2014 08:40 PM, Gabriel Hurley wrote:
 we could have a poll for the name.
 Gabriel, would you like to run that?
 
 I can run that if you like, though it might be more official coming from the 
 PTL. ;-)

I'm sure that David is reading this and that he will tell us when
anything we are doing is wrong or could be improved. I think we should
do whatever needs doing and not bother the PTL with every triviality.

It would be great if you could do it, because I'm not good at this kind
of stuff, and I just want to have the name. I do realize the choice is
important as well as the way it was made, but I'm hopeless at it.

It would be a great help if  you could handle that.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon][infra] Plan for the splitting of Horizon into two repositories

[Radomir Dopieralski]

On 05/28/2014 10:03 PM, Gabriel Hurley wrote:
 It's sort of a silly point, but as someone who would likely consume the
 split-off package outside of the OpenStack context, please give it a
proper
 name instead of django_horizon. The module only works in Django, the
name
 adds both clutter and confusion, and it's against the Django
 community's conventions to have the python import name be prefixed
with django_.

Since the name is completely irrelevant from the technical point of
view, and everybody will naturally have their own opinions about it,
I would prefer to skip the whole discussion and just settle on the most
mundane, boring, uninspired and obvious name that we can use, just to
avoid bikeshedding and surprises.

I didn't realize it would violate Django community's conventions, can
you point to where they are documented?

I don't think this is important, but since we have some time until the
patches for static files and other stuff clear, we could have a poll for
the name. Gabriel, would you like to run that?

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon][infra] Plan for the splitting of Horizon into two repositories

[Radomir Dopieralski]

Hello,

we plan to finally do the split in this cycle, and I started some
preparations for that. I also started to prepare a detailed plan for the
whole operation, as it seems to be a rather big endeavor.

You can view and amend the plan at the etherpad at:
https://etherpad.openstack.org/p/horizon-split-plan

It's still a little vague, but I plan to gradually get it more detailed.
All the points are up for discussion, if anybody has any good ideas or
suggestions, or can help in any way, please don't hesitate to add to
this document.

We still don't have any dates or anything -- I suppose we will work that
out soonish.

Oh, and great thanks to all the people who have helped me so far with
it, I wouldn't even dream about trying such a thing without you. Also
thanks in advance to anybody who plans to help!

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] Static file handling -- followup

[Radomir Dopieralski]

Hello,

this is a followup on the design session we had at the meeting about
the handling of static files. You can see the etherpad from that session
here: https://etherpad.openstack.org/p/juno-summit-horizon-static-files


The split:

We are going to use rather uninspired, but very clear and well-known
names. The horizon (library) part is going to be named
django-horizon, and the openstack_dashboard is going to be named
horizon. We will clone the horizon repository as django-horizon
soon(ish) and start removing the unwanted files from both of them
-- this way we will preserve the history.


The JavaScript libraries unbundling:

I'm packaging all the missing libraries, except for Angular.js, as
XStatic packages:

https://pypi.python.org/pypi/XStatic-D3
https://pypi.python.org/pypi/XStatic-Hogan
https://pypi.python.org/pypi/XStatic-JSEncrypt
https://pypi.python.org/pypi/XStatic-QUnit
https://pypi.python.org/pypi/XStatic-Rickshaw
https://pypi.python.org/pypi/XStatic-Spin

There is also a patch for unbundling JQuery:
https://review.openstack.org/#/c/82516/
And the corresponding global requirements for it:
https://review.openstack.org/#/c/94337/

Once it is in, I will prepare a patch with the rest of libraries.

We will also unbundle Bootstrap, but first we need to deal with its
compilation, read below.


The style files compilation:

We are going to go with PySCSS compiler, plus django-pyscss. The
proof-of-concept patch has been rebased and updated, and is waiting
for your reviews: https://review.openstack.org/#/c/90371/
It is also waiting for adding the needed libraries to the global
requirements: https://review.openstack.org/#/c/94376/


The style files dependencies and pluggability:

Turns out that we don't have to generate a file with all the includes
after all, because django-pyscss actually solves that problem for us.
Horizon's plugins can refer to Horizon's own files easily now.


The linter and other tools:

We will be able to include the linter in the gate check without having
to explicitly depend on it in Horizon itself.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Tuskar][Horizon] Javascript linter

[Radomir Dopieralski]

On 02/04/14 15:26, Kevin Conway wrote:
 What licensing issues were brought up that prevent the use of JSLint or
 JSHint? Both are MIT licensed.
 
 Granted, JSLint has an additional clause:
 
 The Software shall be used for Good, not Evil.
 
 
 Maybe that's it? If so, Crockford has been known to make exceptions for
 organizations that wish to use his code for potentially evil
 purposes: 
 https://www.youtube.com/watch?v=-C-JoyNuQJsfeature=player_detailpage#t=2480s.

Yes, that's exactly it. An exception is not enough -- that clause simply
makes that license incompatible with OpenStack's license. To use it, we
would need to change OpenStack's license too, and it quickly becomes
quite complex.

You have to remember that organizations like NSA use OpenStack, so we
can't possibly include that clause in its license ;)

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] xstatic for removing bundled js libs

[Radomir Dopieralski]

On 26/03/14 11:43, Maxime Vidori wrote:
[...]
 I propose a more general solution of using xstatic-* Python packages,
 which contain basically just the files that we want to bundle, plus some
 small amount of metadata. All of the JavaScript (and any static files,
 really, including styles, fonts and icons) would be then handled the
 same way, by just adding the relevant package to the requirements and to
 the settings.py file. Packaging the libraries that are missing is very
 easy (as there is no extra code to write), and we get to share the
 effort with other projects that use xstatic.
[...]

 Hi Radomir,
 
 I quickly looked at xstatic and I have the impression that you have to handle
 manually the retrievement of the javascript library files and the version of
 the scripts. What do you think of wrapping bower in a script in order to
 dynamically generate packages with the right versions?

How the actual xstatic-* packages are created and maintained is up to
the packager. They are not part of Horizon, so you can use any tools
you wish to create and update them.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][db][performance] Proposal: Get rid of soft deletion (step by step)

[Radomir Dopieralski]

On 16/03/14 06:04, Clint Byrum wrote:

 I think you can achieve this level of protection simply by denying
 interactive users the rights to delete individual things directly, and
 using stop instead of delete. Then have something else (cron?) clean up
 stopped instances after a safety period has been reached.

I would be very interested in the approach to determining the optimal
value for that safety period you are mentioning. Or is this going to be
left as an exercise for the reader? (That is, set in the configuration,
so that the users have to somehow perform this impossible task.)

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][db][performance] Proposal: Get rid of soft deletion (step by step)

[Radomir Dopieralski]

Hello,

I also think that this thread is going in the wrong direction, but I
don't think the direction Boris wants is the correct one either. Frankly
I'm a little surprised that nobody mentioned another advantage that soft
delete gives us, the one that I think it was actually used for originally.

You see, soft delete is an optimization. It's there to make the system
work faster as a whole, have less code and be simpler to maintain and debug.

How does it do it, when, as clearly shown in the first post in this
thread, it makes the queries slower, requires additional indices in the
database and more logic in the queries? The answer is, by doing more
with those queries, by making you write less code, execute fewer queries
to the databases and avoid duplicating the same data in multiple places.

OpenStack is a big, distributed system of multiple databases that
sometimes rely on each other and cross-reference their records. It's not
uncommon to have some long-running operation started, that uses some
data, and then, in the middle of its execution, have that data deleted.
With soft delete, that's not a problem -- the operation can continue
safely and proceed as scheduled, with the data it was started with in
the first place -- it still has access to the deleted records as if
nothing happened. You simply won't be able to schedule another operation
like that with the same data, because it has been soft-deleted and won't
pass the validation at the beginning (or even won't appear in the UI or
CLI). This solves a lot of race conditions, error handling, additional
checks to make sure the record still exists, etc.

Without soft delete, you need to write custom code every time to handle
the case of a record being deleted mid-operation, including all the
possible combinations of which record and when. Or you need to copy all
the relevant data in advance over to whatever is executing that
operation. This cannot be abstracted away entirely (although tools like
TaskFlow help), as this is specific to the case you are handling. And
it's not easy to find all the places where you can have a race condition
like that -- especially when you are modifying existing code that has
been relying on soft delete before. You can have bugs undetected for
years, that only appear in production, on very large deployments, and
are impossible to reproduce reliably.

There are more similar cases like that, including cascading deletes and
more advanced stuff, but I think this single case already shows that
the advantages of soft delete out-weight its disadvantages.

On 13/03/14 19:52, Boris Pavlovic wrote:
 Hi all, 
 
 
 I would like to fix direction of this thread. Cause it is going in wrong
 direction. 
 
 To assume:
 1) Yes restoring already deleted recourses could be useful. 
 2) Current approach with soft deletion is broken by design and we should
 get rid of them. 
 
 More about why I think that it is broken: 
 1) When you are restoring some resource you should restore N records
 from N tables (e.g. VM)
 2) Restoring sometimes means not only restoring DB records. 
 3) Not all resources should be restorable (e.g. why I need to restore
 fixed_ip? or key-pairs?)
 
 
 So what we should think about is:
 1) How to implement restoring functionally in common way (e.g. framework
 that will be in oslo) 
 2) Split of work of getting rid of soft deletion in steps (that I
 already mention):
 a) remove soft deletion from places where we are not using it
 b) replace internal code where we are using soft deletion to that framework 
 c) replace API stuff using ceilometer (for logs) or this framework (for
 restorable stuff)
 
 
 To put in a nutshell: Restoring Delete resources / Delayed Deletion !=
 Soft deletion. 
 
 
 Best regards,
 Boris Pavlovic 
 
 
 
 On Thu, Mar 13, 2014 at 9:21 PM, Mike Wilson geekinu...@gmail.com
 mailto:geekinu...@gmail.com wrote:
 
 For some guests we use the LVM imagebackend and there are times when
 the guest is deleted on accident. Humans, being what they are, don't
 back up their files and don't take care of important data, so it is
 not uncommon to use lvrestore and undelete an instance so that
 people can get their data. Of course, this is not always possible if
 the data has been subsequently overwritten. But it is common enough
 that I imagine most of our operators are familiar with how to do it.
 So I guess my saying that we do it on a regular basis is not quite
 accurate. Probably would be better to say that it is not uncommon to
 do this, but definitely not a daily task or something of that ilk.
 
 I have personally undeleted an instance a few times after
 accidental deletion also. I can't remember the specifics, but I do
 remember doing it :-).
 
 -Mike
 
 
 On Tue, Mar 11, 2014 at 12:46 PM, Johannes Erdfelt
 johan...@erdfelt.com mailto:johan...@erdfelt.com wrote:
 
 On Tue, Mar 11, 2014, Mike Wilson geekinu...@gmail.com
 

Re: [openstack-dev] [all][db][performance] Proposal: Get rid of soft deletion (step by step)

[Radomir Dopieralski]

On 14/03/14 11:08, Alexei Kornienko wrote:
 On 03/14/2014 09:37 AM, Radomir Dopieralski wrote:

[snip]

 OpenStack is a big, distributed system of multiple databases that
 sometimes rely on each other and cross-reference their records. It's not
 uncommon to have some long-running operation started, that uses some
 data, and then, in the middle of its execution, have that data deleted.
 With soft delete, that's not a problem -- the operation can continue
 safely and proceed as scheduled, with the data it was started with in
 the first place -- it still has access to the deleted records as if
 nothing happened. You simply won't be able to schedule another operation
 like that with the same data, because it has been soft-deleted and won't
 pass the validation at the beginning (or even won't appear in the UI or
 CLI). This solves a lot of race conditions, error handling, additional
 checks to make sure the record still exists, etc.

 1) Operation in SQL are working in transactions so deleted records will
 be visible for other clients until transaction commit.

 2) If someone inside the same transaction will try to use record that is
 already deleted it's definitely an error in our code and should be fixed.
 I don't think that such use case can be used as an argument to keep soft
 deleted records.

Yes, that's why it works just fine when you have a single database in
one place. You can have locks, transactions, cascading operations and
all this stuff, and you have a guarantee that you are always in a
consistent state, unless there is a horrible bug.

OpenStack, however, is not a single database. There is no system-wide
solution for locks, transactions or rollbacks. Every time you reference
anything across databases, you are going to run into this problem.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Edit subnet in workflows - ip_version hidden?

[Radomir Dopieralski]

On 11/03/14 16:57, Abishek Subramanian (absubram) wrote:

 Althouh - how up to date is this code?

This should be easy to check with the git blame command:

$ git blame
openstack_dashboard/dashboards/project/networks/subnets/workflows.py

[...]
31d55e50 (Akihiro MOTOKI  2013-01-04 18:33:03 +0900  56) class
CreateSubnet(network_workflows.CreateNetwork):
[...]
31d55e50 (Akihiro MOTOKI  2013-01-04 18:33:03 +0900  82) class
UpdateSubnetInfoAction(CreateSubnetInfoAction):
[...]
31d55e50 (Akihiro MOTOKI  2013-01-04 18:33:03 +0900 101)
#widget=forms.Select(
[...]

As you can see, it's all in the same patch, so it's on purpose.

It seems to me, that in the update dialog you are not supposed to change
the IP Version field, Akihiro Motoki tried to disable it
first, but then he hit the problem with the browser not submitting
the field's value and the form displaying the wrong option in there,
so he decided to hide it instead. But we won't know until the author
speaks for himself.

Personally, I would also add a check in the clean() method that the
IP Version field value indeed didn't change -- to make sure nobody
edited the form's HTML to get rid of the disabled or readonly attribute.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Edit subnet in workflows - ip_version hidden?

[Radomir Dopieralski]

On 11/03/14 15:52, Abishek Subramanian (absubram) wrote:
 Hi,
 
 I had a question regarding the
 dashboards/project/networks/subnets/workflows.py
 file and in particular the portion of the ip_version field.
 
 It is marked as a hidden input field for the update subnet class with this
 note.
 
 # NOTE(amotoki): When 'disabled' attribute is set for the ChoiceField
 # and ValidationError is raised for POST request, the initial value of
 # the ip_version ChoiceField is not set in the re-displayed form
 # As a result, 'IPv4' is displayed even when IPv6 is used if
 # ValidationError is detected. In addition 'required=True' check
 complains
 # when re-POST since the value of the ChoiceField is not set.
 # Thus now I use HiddenInput for the ip_version ChoiceField as a work
 # around.
 
 
 
 Can I get a little more context to this please?
 I'm not sure I understand why it says this field always is displayed as
 IPv4.
 Is this still the case? Adding some debug logs I seem to see that the
 ipversion is correctly being detected as 4 or 6 as the case may be.

Some browsers (Chrome, iirc) will not submit the values from form fields
that are disabled. That means, that when re-displaying this form
(after an error in any other field, for example), that field's value
will be missing, and the browser will happily display the first option,
which is ipv4.

Another solution could be perhaps using readonly instead of disabled.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Tuskar] Dealing with passwords in Tuskar-API

[Radomir Dopieralski]

On 21/02/14 10:38, Dougal Matthews wrote:
 On 21/02/14 09:24, Tomas Sedovic wrote:
 On 20/02/14 16:24, Imre Farkas wrote:
 On 02/20/2014 03:57 PM, Tomas Sedovic wrote:
 On 20/02/14 15:41, Radomir Dopieralski wrote:
 On 20/02/14 15:00, Tomas Sedovic wrote:

 Are we even sure we need to store the passwords in the first
 place? All
 this encryption talk seems very premature to me.

 How are you going to redeploy without them?


 What do you mean by redeploy?

 1. Deploy a brand new overcloud, overwriting the old one
 2. Updating the services in the existing overcloud (i.e. image updates)
 3. Adding new machines to the existing overcloud
 4. Autoscaling
 5. Something else
 6. All of the above

 I'd guess each of these have different password workflow requirements.

 I am not sure if all these use cases have different password
 requirement. If you check devtest, no matter whether you are creating or
 just updating your overcloud, all the parameters have to be provided for
 the heat template:
 https://github.com/openstack/tripleo-incubator/blob/master/scripts/devtest_overcloud.sh#L125



 I would rather not require the user to enter 5/10/15 different passwords
 every time Tuskar updates the stack. I think it's much better to
 autogenerate the passwords for the first time, provide an option to
 override them, then save and encrypt them in Tuskar. So +1 for designing
 a proper system for storing the passwords.

 Well if that is the case and we can't change the templates/heat to
 change that, the secrets should be put in Keystone or at least go
 through Keystone. Or use Barbican or whatever.

 We shouldn't be implementing crypto in Tuskar.
 
 +1, after giving this quite a bit of thought i completely agree. This
 is really out of the scope of Tuskar. I think we should definitely go
 down this route and only fall back to storing all these details
 later, that could be an improvement if it turns out to be a real
 usability problem.
 
 At the moment we are guessing, we don't even know how many passwords
 we need to store. So we should progress with the safest and simplest
 option (which is to not store them) and consider changing this later.

I think we have a pretty good idea:
https://github.com/openstack/tuskar-ui/blob/master/tuskar_ui/infrastructure/overcloud/workflows/undeployed_configuration.py#L23-L222

(just count the NoEcho: true lines)
-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Tuskar] Dealing with passwords in Tuskar-API

[Radomir Dopieralski]

On 19/02/14 18:29, Dougal Matthews wrote:
 The question for me, is what passwords will we have and when do we need
 them? Are any of the passwords required long term.

We will need whatever the Heat template needs to generate all the
configuration files. That includes passwords for all services that are
going to be configured, such as, for example, Swift or MySQL.

I'm not sure about the exact mechanisms in Heat, but I would guess that
we will need all the parameters, including passwords, when the templates
are re-generated. We could probably generate new passwords every time,
though.

 If we do need to store passwords it becomes a somewhat thorny issue, how
 does Tuskar know what a password is? If this is flagged up by the
 UI/client then we are relying on the user to tell us which isn't wise.

All the template parameters that are passwords are marked in the Heat
parameter list that we get from it as NoEcho: true, so we do have an
idea about which parts are sensitive.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Tuskar] Dealing with passwords in Tuskar-API

[Radomir Dopieralski]

On 20/02/14 11:21, Dougal Matthews wrote:
 If we do store passwords however, I wonder if we are
 best to encrypt everything to be safe. The overhead shouldn't be that
 big and it may be better than special casing the NoEcho values.

I think that before we start encrypting everything, we need to ask
ourselves the question about system boundaries and about what we are
protecting from what. Otherwise we will end up with ridiculous things
like encrypting the passwords and storing the decryption key right in
the same place. In other words, this has to be designed.
-- 
Radomir Dopieralski



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Tuskar] Dealing with passwords in Tuskar-API

[Radomir Dopieralski]

On 20/02/14 12:02, Radomir Dopieralski wrote:
 On 20/02/14 11:46, Dougal Matthews wrote:
 On 20/02/14 10:36, Radomir Dopieralski wrote:
 On 20/02/14 11:21, Dougal Matthews wrote:
 If we do store passwords however, I wonder if we are
 best to encrypt everything to be safe. The overhead shouldn't be that
 big and it may be better than special casing the NoEcho values.

 I think that before we start encrypting everything, we need to ask
 ourselves the question about system boundaries and about what we are
 protecting from what. Otherwise we will end up with ridiculous things
 like encrypting the passwords and storing the decryption key right in
 the same place. In other words, this has to be designed.

 Absolutely. I couldn't agree more and hope I didn't suggest otherwise :)
 
 So what is the smallest subsystem (or subsystems) that needs those
 passwords, and what storage it has access to that other subsystems
 don't, so we could put the key there?
 
 As I see it, the passwords are needed by:
 
 * Heat, for generating the templates (may be passed from Tuskar-API),
 * Tuskar-API, for passing them to Heat and for registering services in
   Keystone,
 * Tuskar-UI, if we want to display them to the user on request (do
   we?), may also be passed from Tuskar-API.
 
 What is the storage that Tuskar-API has access to, but other parts
 don't? I would guess that's the Tuskar database. But that's where
 the passwords are already stored, so storing the key there doesn't
 make much sense. Anybody who gets access to Tuskar-API gets the
 passwords, whether we encrypt them or not. Anybody who doesn't have
 access to Tuskar-API doesn't get the passwords, whether we encrypt
 them or not.
 
Thinking about it some more, all the uses of the passwords come as a
result of an action initiated by the user either by tuskar-ui, or by
the tuskar command-line client. So maybe we could put the key in their
configuration and send it with the request to (re)deploy. Tuskar-API
would still need to keep it for the duration of deployment (to register
the services at the end), but that's it.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Tuskar] Dealing with passwords in Tuskar-API

[Radomir Dopieralski]

On 20/02/14 14:10, Jiří Stránský wrote:
 On 20.2.2014 12:18, Radomir Dopieralski wrote:

 Thinking about it some more, all the uses of the passwords come as a
 result of an action initiated by the user either by tuskar-ui, or by
 the tuskar command-line client. So maybe we could put the key in their
 configuration and send it with the request to (re)deploy. Tuskar-API
 would still need to keep it for the duration of deployment (to register
 the services at the end), but that's it.
 
 This would be possible, but it would damage the user experience quite a
 bit. Afaik other deployment tools solve password storage the same way we
 do now.

I don't think it would damage the user experience so much. All you need
is an additional configuration option in Tuskar-UI and Tuskar-client,
the encryption key.

That key would be used to encrypt the passwords when they are first sent
to Tuskar-API, and also added to the (re)deployment calls.

This way, if the database leaks due to a security hole in MySQL or bad
engineering practices administering the database, the passwords are
still inaccessible. To get them, the attacker would need to get
*both* the database and the config files from host on which Tuskar-UI runs.

With the tuskar-client it's a little bit more obnoxious, because you
would need to configure it on every host from which you want to use it,
but you already need to do some configuration to point it at the
tuskar-api and authenticate it, so it's not so bad.

I agree that this complicates the whole process a little, and adds
another potential failure point though.
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Tuskar] Dealing with passwords in Tuskar-API

[Radomir Dopieralski]

On 20/02/14 15:00, Tomas Sedovic wrote:

 Are we even sure we need to store the passwords in the first place? All
 this encryption talk seems very premature to me.

How are you going to redeploy without them?
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Tuskar] Dealing with passwords in Tuskar-API

[Radomir Dopieralski]

On 20/02/14 15:57, Tomas Sedovic wrote:
 On 20/02/14 15:41, Radomir Dopieralski wrote:
 On 20/02/14 15:00, Tomas Sedovic wrote:

 Are we even sure we need to store the passwords in the first place? All
 this encryption talk seems very premature to me.

 How are you going to redeploy without them?

 
 What do you mean by redeploy?
 
 1. Deploy a brand new overcloud, overwriting the old one
 2. Updating the services in the existing overcloud (i.e. image updates)
 3. Adding new machines to the existing overcloud
 4. Autoscaling
 5. Something else
 6. All of the above

I mean clicking scale in tuskar-ui.
-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] why do we put a license in every file?

[Radomir Dopieralski]

On 05/02/14 17:46, Jay Pipes wrote:
 On Wed, 2014-02-05 at 16:29 +, Greg Hill wrote:
 I'm new, so I'm sure there's some history I'm missing, but I find it bizarre 
 that we have to put the same license into every single file of source code 
 in our projects.
 
 Meh, probably just habit and copy/paste behavior.

It's actually not just a habit, it's a requirement, and hacking has a
check for files with missing licenses.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] RFC - Suggestion for switching from Less to Sass (Bootstrap 3 Sass support)

[Radomir Dopieralski]

On 05/02/14 17:32, Jaromir Coufal wrote:
[snip]

 Other concern will be with compilers. So far I've found 3 ways:
 * rails dependency (how big problem would it be?)
 * https://pypi.python.org/pypi/scss/0.7.1
 * https://pypi.python.org/pypi/SassPython/0.2.1
 * ... (other suggestions?)

The first of those links actually points to a newer and better
maintained project: https://github.com/Kronuz/pyScss

The question is, are we jumping out of the frying pan into fire? Are we
going to have exactly the same problems a couple of months or years down
the road with SASS? How healthy is the SASS community compared to that
of LESS and how good are the available tools?

We have to remember that we are adding a burden of learning new language
for all developers -- although SASS and LESS are relatively similar, we
are still running the risk of losing some contributors who don't want to
jump through one more hoop.

Finally, I wonder how well it would work with multiple SASS files
collected from different django apps. Currently, all the LESS files for
Bootstrap are included in Openstack Dashboard, simply because the
dashboard's files need to inherit from them, and putting them in Horizon
would make that impossible. It's a workaround, but it also makes it
impossible for apps other than the dashboard (like tuskar-ui) to inherit
from those less files. I think that if we had a solution for that for
SASS, that would be another strong advantage of switching.
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev