Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-25 Thread Thomas Goirand
On 11/22/2014 04:39 AM, Fox, Kevin M wrote:
 Simply having a git repository does not imply that its source.
 
 In fact, if its considered compiled (minified), I'm thinking the debian rules 
 would prevent sourcing from it?
 
 Thanks,
 Kevin

Indeed, we don't package minified sources.

Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-25 Thread Thomas Goirand
On 11/21/2014 08:31 PM, Donald Stufft wrote:
 
 On Nov 21, 2014, at 3:59 AM, Thomas Goirand z...@debian.org wrote:


 I'm not sure I understand the meaning behind this question. bower
 install angular downloads a bower package called angular.

 Isn't there is a simple URL that I may use with wget? I don't really
 want to use bower directly, I just would like to have a look to the
 content of the bower package.
 
 You can’t. Bower doesn’t have “traditional” packages where you take a
 directory and archive it using tar/zip/whatever and then upload it to
 some repo. Bower has a registry which maps names to git URLs and then
 the bower CLI looks up that mapping, fetches the git repository and then
 uses that as the input to the “look at metadata and do stuff with files”
 part of the package manager instead of the output of an un-unarchival
 command.

Then this makes Bower a very bad candidate to debianize stuff. We'll
have a moving target with a constantly changing git from upstream,
meaning that we'll have all sorts of surprise in the gate.

Frankly, this Bower thing scares me, and I don't really understand why
we're not continuing to use XStatic stuff, which was really convenient
and has been proven to work during the Juno cycle.

Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-25 Thread Monty Taylor
On 11/25/2014 03:40 PM, Richard Jones wrote:
 On Wed Nov 26 2014 at 3:36:27 AM Thomas Goirand z...@debian.org wrote:
 
 On 11/21/2014 08:31 PM, Donald Stufft wrote:

 On Nov 21, 2014, at 3:59 AM, Thomas Goirand z...@debian.org wrote:


 I'm not sure I understand the meaning behind this question. bower
 install angular downloads a bower package called angular.

 Isn't there is a simple URL that I may use with wget? I don't really
 want to use bower directly, I just would like to have a look to the
 content of the bower package.

 You can’t. Bower doesn’t have “traditional” packages where you take a
 directory and archive it using tar/zip/whatever and then upload it to
 some repo. Bower has a registry which maps names to git URLs and then
 the bower CLI looks up that mapping, fetches the git repository and then
 uses that as the input to the “look at metadata and do stuff with files”
 part of the package manager instead of the output of an un-unarchival
 command.

 Then this makes Bower a very bad candidate to debianize stuff. We'll
 have a moving target with a constantly changing git from upstream,
 meaning that we'll have all sorts of surprise in the gate.

 
 It's no more constantly moving than any other project. Bower versions are
 tied to git tags. In fact, since debian etc. usually go to the repository
 rather than use release tarballs, bower *improves* things by requiring the
 tags, making it easier for you to isolate the version in the repository
 that you need, whereas otherwise people just have to remember to tag and
 often don't :)
 
 
 
 Frankly, this Bower thing scares me, and I don't really understand why
 we're not continuing to use XStatic stuff, which was really convenient
 and has been proven to work during the Juno cycle.

 
 We're doing this so we don't have the additional burden of creating and
 maintaining the xstatic packages.

Also, it's _very_ standard javascript tooling. It's what javascript devs
use. So sooner or later there's going to need to be a proper story for
it in Debian if Debian wants to continue to be able to provide value
around applications written in javascript. Might as well be the
trailblazers, no?


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-21 Thread Thomas Goirand
On 11/21/2014 01:51 PM, Richard Jones wrote:
 On 21 November 2014 16:12, Thomas Goirand z...@debian.org
 mailto:z...@debian.org wrote:
 
 On 11/21/2014 10:52 AM, Richard Jones wrote:
  On 11/18/2014 04:22 PM, Radomir Dopieralski wrote:
   If we use Bower, we don't need to use Xstatic. It would be pure
   overhead. Bower already takes care of tracking releases and 
 versions,
   and of bundling the files. All we need is a simple line in the
   settings.py telling Django where it puts all the files -- we don't
   really need Xstatic just for that. The packagers can then take 
 those
   Bower packages and turn them into system packages, and just 
 add/change
   the paths in settings.py to where they put the files. All in one
   place.
 
  The issue is that there's often not just a single path, but a full
  directory structure to address. That is easily managed with a Debian
  xstatic package, not sure how it would be with Bower.
 
 
  I'm not sure what the difference is (unless it's just related to the
  Debian-specific historical issue you raise below). xstatic and bower are
  remarkably similar a directory to be packaged and some meta-data
  describing it.
 
 Let me explain again then.
 
 Let's say there's python-xstatic-foo, and libjs-foo in Debian. If the
 directory structure of libjs-foo is very different from xstatic-foo, I
 can address that issue with symlinks within the xstatic package. Just
 changing the BASE_DIR may not be enough, as libjs-foo may have files
 organized in a very different way than in the upstream package for foo.
 
 
 OK, so python-xstatic-foo can depend on libjs-foo and just symlink, fair
 enough. I'm still not sure what makes bower unique in this respect,

I was under the impression that I wouldn't be able to do the same
symlink thing with Bower. If I am, then great!

 although it'd be nice to avoid creating additional packages just to
 symlink things around for bower, which I think is what you're getting at.

Just to make sure: we're not moving away from the current already
existing xstatic packages are we?

Also, yes, if I can avoid to have a bower package, that'd be great. But
not sure how. It worked with the XStatic packages, and my main concern
about switching to bower is exactly that: how is it going to work
compared to XStatic stuff.

  Again; bower is not npm! Jasmine is a command-line program which is
  packaged by npm. Bower packages bundles of stuff that are included in
  web applications. bower itself, a command-line tool, is packaged by npm,
  but itself manages other packages which are not command-line things, but
  just bundles of css, javascript, images, fonts, etc. that are resources
  for web applications to use.
 
 Sure. But how do I download a bower package then?
 
 I'm not sure I understand the meaning behind this question. bower
 install angular downloads a bower package called angular.

Isn't there is a simple URL that I may use with wget? I don't really
want to use bower directly, I just would like to have a look to the
content of the bower package.

Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-21 Thread Radomir Dopieralski
On 21/11/14 06:12, Thomas Goirand wrote:

 Let's say there's python-xstatic-foo, and libjs-foo in Debian. If the
 directory structure of libjs-foo is very different from xstatic-foo, I
 can address that issue with symlinks within the xstatic package. Just
 changing the BASE_DIR may not be enough, as libjs-foo may have files
 organized in a very different way than in the upstream package for foo.

You can do it without xstatic even more easily, without having to muck
around with symlinks. Django has this STATICFILES_DIRS setting variable,
that you can set, that lets you specify on what URL each static
directory should be available. For instance, I use it currently to work
around the issue with jquery-ui changing directory structure between
versions:

if xstatic.main.XStatic(xstatic.pkg.jquery_ui).version.startswith('1.10.'):
# The 1.10.x versions already contain the 'ui' directory.
STATICFILES_DIRS.append(
('horizon/lib/jquery-ui',
 xstatic.main.XStatic(xstatic.pkg.jquery_ui).base_dir))
else:
# Newer versions dropped the directory, add it to keep the path the
same.
STATICFILES_DIRS.append(
('horizon/lib/jquery-ui/ui',
 xstatic.main.XStatic(xstatic.pkg.jquery_ui).base_dir))

But in your Debian package, you can completely drop xstatic and just use
absolute paths to the filesystem explicitly.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-21 Thread Donald Stufft

 On Nov 21, 2014, at 3:59 AM, Thomas Goirand z...@debian.org wrote:
 
 
 I'm not sure I understand the meaning behind this question. bower
 install angular downloads a bower package called angular.
 
 Isn't there is a simple URL that I may use with wget? I don't really
 want to use bower directly, I just would like to have a look to the
 content of the bower package.

You can’t. Bower doesn’t have “traditional” packages where you take a
directory and archive it using tar/zip/whatever and then upload it to
some repo. Bower has a registry which maps names to git URLs and then
the bower CLI looks up that mapping, fetches the git repository and then
uses that as the input to the “look at metadata and do stuff with files”
part of the package manager instead of the output of an un-unarchival
command.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-21 Thread Jeremy Stanley
On 2014-11-21 07:31:36 -0500 (-0500), Donald Stufft wrote:
 You can’t. Bower doesn’t have “traditional” packages where you take a
 directory and archive it using tar/zip/whatever and then upload it to
 some repo. Bower has a registry which maps names to git URLs and then
 the bower CLI looks up that mapping, fetches the git repository and then
 uses that as the input to the “look at metadata and do stuff with files”
 part of the package manager instead of the output of an un-unarchival
 command.

This raises interesting free software philosophy/license
questions... how do I redistribute (or even examine) the source of
a bower-managed package? Is there a way without actually
reverse-engineering the toolchain?
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-21 Thread Donald Stufft

 On Nov 21, 2014, at 11:32 AM, Jeremy Stanley fu...@yuggoth.org wrote:
 
 On 2014-11-21 07:31:36 -0500 (-0500), Donald Stufft wrote:
 You can’t. Bower doesn’t have “traditional” packages where you take a
 directory and archive it using tar/zip/whatever and then upload it to
 some repo. Bower has a registry which maps names to git URLs and then
 the bower CLI looks up that mapping, fetches the git repository and then
 uses that as the input to the “look at metadata and do stuff with files”
 part of the package manager instead of the output of an un-unarchival
 command.
 
 This raises interesting free software philosophy/license
 questions... how do I redistribute (or even examine) the source of
 a bower-managed package? Is there a way without actually
 reverse-engineering the toolchain?

Well it’s a git repository, so you could just clone it and look at it.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-21 Thread Jeremy Stanley
On 2014-11-21 11:39:00 -0500 (-0500), Donald Stufft wrote:
 Well it’s a git repository, so you could just clone it and look at
 it.

Aha, your earlier description made it sound like Bower was a file
registry mapping to various random contents from a bunch of revision
control repositories to assemble any one package. If Bower packages
generally map back to one repository per package (even if there are
multiple packages per repository) then that seems much more sane to
deal with.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-21 Thread Donald Stufft

 On Nov 21, 2014, at 11:57 AM, Jeremy Stanley fu...@yuggoth.org wrote:
 
 On 2014-11-21 11:39:00 -0500 (-0500), Donald Stufft wrote:
 Well it’s a git repository, so you could just clone it and look at
 it.
 
 Aha, your earlier description made it sound like Bower was a file
 registry mapping to various random contents from a bunch of revision
 control repositories to assemble any one package. If Bower packages
 generally map back to one repository per package (even if there are
 multiple packages per repository) then that seems much more sane to
 deal with.
 -- 
 Jeremy Stanley
 

Yea sorry, the bower registry (aka bower PyPI) is a mapping of name: git URL.


---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-21 Thread Fox, Kevin M
Simply having a git repository does not imply that its source.

In fact, if its considered compiled (minified), I'm thinking the debian rules 
would prevent sourcing from it?

Thanks,
Kevin

From: Donald Stufft [don...@stufft.io]
Sent: Friday, November 21, 2014 8:39 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Horizon] the future of angularjs development  
in Horizon

 On Nov 21, 2014, at 11:32 AM, Jeremy Stanley fu...@yuggoth.org wrote:

 On 2014-11-21 07:31:36 -0500 (-0500), Donald Stufft wrote:
 You can’t. Bower doesn’t have “traditional” packages where you take a
 directory and archive it using tar/zip/whatever and then upload it to
 some repo. Bower has a registry which maps names to git URLs and then
 the bower CLI looks up that mapping, fetches the git repository and then
 uses that as the input to the “look at metadata and do stuff with files”
 part of the package manager instead of the output of an un-unarchival
 command.

 This raises interesting free software philosophy/license
 questions... how do I redistribute (or even examine) the source of
 a bower-managed package? Is there a way without actually
 reverse-engineering the toolchain?

Well it’s a git repository, so you could just clone it and look at it.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-21 Thread Richard Jones
All the bower components I've used have included full source alongside any
minified versions.

On Sat, 22 Nov 2014 07:40 Fox, Kevin M kevin@pnnl.gov wrote:

 Simply having a git repository does not imply that its source.

 In fact, if its considered compiled (minified), I'm thinking the debian
 rules would prevent sourcing from it?

 Thanks,
 Kevin
 
 From: Donald Stufft [don...@stufft.io]
 Sent: Friday, November 21, 2014 8:39 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Horizon] the future of angularjs
 development  in Horizon

  On Nov 21, 2014, at 11:32 AM, Jeremy Stanley fu...@yuggoth.org wrote:
 
  On 2014-11-21 07:31:36 -0500 (-0500), Donald Stufft wrote:
  You can’t. Bower doesn’t have “traditional” packages where you take a
  directory and archive it using tar/zip/whatever and then upload it to
  some repo. Bower has a registry which maps names to git URLs and then
  the bower CLI looks up that mapping, fetches the git repository and then
  uses that as the input to the “look at metadata and do stuff with files”
  part of the package manager instead of the output of an un-unarchival
  command.
 
  This raises interesting free software philosophy/license
  questions... how do I redistribute (or even examine) the source of
  a bower-managed package? Is there a way without actually
  reverse-engineering the toolchain?

 Well it’s a git repository, so you could just clone it and look at it.

 ---
 Donald Stufft
 PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-20 Thread Thomas Goirand
On 11/17/2014 06:54 PM, Radomir Dopieralski wrote:
 - A tool, probably a script, that would help packaging the Bower
 packages into DEB/RPM packages. I suspect the Debian/Fedora packagers
 already have a semi-automatic solution for that.

Nop. Bower isn't even packaged in Debian. Though I may try to do it
(when I'm done with other Mirantis stuff like packaging Fuel for Debian...).

On 11/18/2014 07:59 AM, Richard Jones wrote:
 I was envisaging us creating a tool which generates xstatic packages
 from bower packages. I'm not the first to think along these
 lines
http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

I think that's a very good idea!

 I wrote the tool today, and you can find it here:

 https://github.com/r1chardj0n3s/flaming-shame

AWESOME ! :)
Then now, everyone is happy. Thank you.

On 11/18/2014 04:22 PM, Radomir Dopieralski wrote:
 If we use Bower, we don't need to use Xstatic. It would be pure
 overhead. Bower already takes care of tracking releases and versions,
 and of bundling the files. All we need is a simple line in the
 settings.py telling Django where it puts all the files -- we don't
 really need Xstatic just for that. The packagers can then take those
 Bower packages and turn them into system packages, and just add/change
 the paths in settings.py to where they put the files. All in one
 place.

The issue is that there's often not just a single path, but a full
directory structure to address. That is easily managed with a Debian
xstatic package, not sure how it would be with Bower.

On 11/18/2014 06:36 PM, Richard Jones wrote:
 I guess I got the message that turning bower packages into system
 packages was something that the Linux packagers were not keen on.

What I'm not a fan of, is that we'll have external dependencies being
bumped all the time, with unexpected consequences. At least, with
xstatic packages, we control what's going on (though I understand the
overhead work problem).

By the way, I went into bower.io as I wanted to have a look. How do I
download a binary package for let's say jasmin? When searching, it just
links to github...

On 11/19/2014 12:14 AM, Radomir Dopieralski wrote:
 We would replace that with:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
os.path.join(BASE_DIR, 'bower_modules/angular'),
 ...
 ]

This would only work if upstream package directory structure is the same
as the one in the distribution. For historical reason, that's
unfortunately often not the case (sometimes because we want to keep
backward compatibility in the distro because of reverse dependencies),
and just changing the path wont make it work.

On 11/19/2014 03:43 AM, Richard Jones wrote:
 +1 to all that, except I'd recommend using django-bower to handle the
 static collection stuff. It's not documented but django-bower has a
 setting BOWER_COMPONENTS_ROOT which would make the above transition much
 simpler. You leave it alone for local dev, and packagers setup
 BOWER_COMPONENTS_ROOT to '/usr/lib/javascript/' or wherever.

s/lib/share/

However, I'm almost sure that wont be enough to make it work. For
example, in Debian, we have /usr/share/javascript/angular.js, not just
/usr/share/javascript/angular. So django-bower would be searching on the
wrong path.

On 11/19/2014 12:25 PM, Richard Jones wrote:
 In their view, bower components don't need to be in global-requirements:

 - there are no other projects that use bower components, so we don't
 need to ensure cross-project compatibility
 - we can vet new versions of bower components as part of standard
 Horizon change review

Maybe that's right for the OpenStack project, but that is a problem at
least for me, as I wont be constantly looking at Horizon dependencies,
just at the global-requirements.txt. So I'm afraid I may miss some new
stuff, and miss the deadline for the next release if I don't pay
attention to it. :(
Anyway, that's not so bad, I can try to adapt, but I just wanted to
raise my concern so that everyone knows about it.

Last thing: I'm currently a bit afraid of what will happen, as I don't
know the tools (bower and such). I wish I had a bit more time to test
them out, but I don't... :( So, I'm just raising my concerns even if
sometimes they may have no root, in the hope that we all find the best
solution that fits everyone. I hope the way I did in this thread is ok.

Cheers,

Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-20 Thread Richard Jones
On Fri Nov 21 2014 at 4:06:51 AM Thomas Goirand z...@debian.org wrote:

 On 11/17/2014 06:54 PM, Radomir Dopieralski wrote:
  - A tool, probably a script, that would help packaging the Bower
  packages into DEB/RPM packages. I suspect the Debian/Fedora packagers
  already have a semi-automatic solution for that.

 Nop. Bower isn't even packaged in Debian. Though I may try to do it
 (when I'm done with other Mirantis stuff like packaging Fuel for
 Debian...).


Just to be clear, it's not bower itself (the command-line tool) that needs
packaging, just the components that bower itself packages.



 On 11/18/2014 07:59 AM, Richard Jones wrote:
  I was envisaging us creating a tool which generates xstatic packages
  from bower packages. I'm not the first to think along these
  lines
 http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

 I think that's a very good idea!

  I wrote the tool today, and you can find it here:
 
  https://github.com/r1chardj0n3s/flaming-shame

 AWESOME ! :)
 Then now, everyone is happy. Thank you.


Well, no, but at least it exists ;)



 On 11/18/2014 04:22 PM, Radomir Dopieralski wrote:
  If we use Bower, we don't need to use Xstatic. It would be pure
  overhead. Bower already takes care of tracking releases and versions,
  and of bundling the files. All we need is a simple line in the
  settings.py telling Django where it puts all the files -- we don't
  really need Xstatic just for that. The packagers can then take those
  Bower packages and turn them into system packages, and just add/change
  the paths in settings.py to where they put the files. All in one
  place.

 The issue is that there's often not just a single path, but a full
 directory structure to address. That is easily managed with a Debian
 xstatic package, not sure how it would be with Bower.


I'm not sure what the difference is (unless it's just related to the
Debian-specific historical issue you raise below). xstatic and bower are
remarkably similar a directory to be packaged and some meta-data describing
it.



 On 11/18/2014 06:36 PM, Richard Jones wrote:
  I guess I got the message that turning bower packages into system
  packages was something that the Linux packagers were not keen on.

 What I'm not a fan of, is that we'll have external dependencies being
 bumped all the time, with unexpected consequences. At least, with
 xstatic packages, we control what's going on (though I understand the
 overhead work problem).


The dependencies won't be bumped any faster than reviewers allow, though I
realise that's not necessarily going to make you sleep any easier. Hmm.



 By the way, I went into bower.io as I wanted to have a look. How do I
 download a binary package for let's say jasmin? When searching, it
 just links to github...


Again; bower is not npm! Jasmine is a command-line program which is
packaged by npm. Bower packages bundles of stuff that are included in web
applications. bower itself, a command-line tool, is packaged by npm, but
itself manages other packages which are not command-line things, but just
bundles of css, javascript, images, fonts, etc. that are resources for web
applications to use.



 On 11/19/2014 12:14 AM, Radomir Dopieralski wrote:
  We would replace that with:
 
  STATICFILES_DIRS = [
  ('horizon/lib/angular',
 os.path.join(BASE_DIR, 'bower_modules/angular'),
  ...
  ]

 This would only work if upstream package directory structure is the same
 as the one in the distribution. For historical reason, that's
 unfortunately often not the case (sometimes because we want to keep
 backward compatibility in the distro because of reverse dependencies),
 and just changing the path wont make it work.

 On 11/19/2014 03:43 AM, Richard Jones wrote:
  +1 to all that, except I'd recommend using django-bower to handle the
  static collection stuff. It's not documented but django-bower has a
  setting BOWER_COMPONENTS_ROOT which would make the above transition much
  simpler. You leave it alone for local dev, and packagers setup
  BOWER_COMPONENTS_ROOT to '/usr/lib/javascript/' or wherever.

 s/lib/share/

 However, I'm almost sure that wont be enough to make it work. For
 example, in Debian, we have /usr/share/javascript/angular.js, not just
 /usr/share/javascript/angular. So django-bower would be searching on the
 wrong path.


That is unfortunate. It may be that Debian therefore has to special-case
angular to handle that case.

I think the general idea of following the component pattern set by bower
(separate directories with no risk of conflicts, and using the bower.json
meta-data to allow automatic configuration of the component) is too good to
dismiss though. Far less work for everyone, including packagers.

Perhaps the new packages should have bower in their name?



 On 11/19/2014 12:25 PM, Richard Jones wrote:
  In their view, bower components don't need to be in global-requirements:
 
  - there are no other projects that use bower components, so we 

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-20 Thread Thomas Goirand
On 11/21/2014 10:52 AM, Richard Jones wrote:
 On 11/18/2014 04:22 PM, Radomir Dopieralski wrote:
  If we use Bower, we don't need to use Xstatic. It would be pure
  overhead. Bower already takes care of tracking releases and versions,
  and of bundling the files. All we need is a simple line in the
  settings.py telling Django where it puts all the files -- we don't
  really need Xstatic just for that. The packagers can then take those
  Bower packages and turn them into system packages, and just add/change
  the paths in settings.py to where they put the files. All in one
  place.
 
 The issue is that there's often not just a single path, but a full
 directory structure to address. That is easily managed with a Debian
 xstatic package, not sure how it would be with Bower.
 
 
 I'm not sure what the difference is (unless it's just related to the
 Debian-specific historical issue you raise below). xstatic and bower are
 remarkably similar a directory to be packaged and some meta-data
 describing it.

Let me explain again then.

Let's say there's python-xstatic-foo, and libjs-foo in Debian. If the
directory structure of libjs-foo is very different from xstatic-foo, I
can address that issue with symlinks within the xstatic package. Just
changing the BASE_DIR may not be enough, as libjs-foo may have files
organized in a very different way than in the upstream package for foo.

 By the way, I went into bower.io http://bower.io as I wanted to
 have a look. How do I
 download a binary package for let's say jasmin? When searching, it
 just links to github...
 
 
 Again; bower is not npm! Jasmine is a command-line program which is
 packaged by npm. Bower packages bundles of stuff that are included in
 web applications. bower itself, a command-line tool, is packaged by npm,
 but itself manages other packages which are not command-line things, but
 just bundles of css, javascript, images, fonts, etc. that are resources
 for web applications to use.

Sure. But how do I download a bower package then?

 This would only work if upstream package directory structure is the same
 as the one in the distribution. For historical reason, that's
 unfortunately often not the case (sometimes because we want to keep
 backward compatibility in the distro because of reverse dependencies),
 and just changing the path wont make it work.
 
 On 11/19/2014 03:43 AM, Richard Jones wrote:
  +1 to all that, except I'd recommend using django-bower to handle the
  static collection stuff. It's not documented but django-bower has a
  setting BOWER_COMPONENTS_ROOT which would make the above
 transition much
  simpler. You leave it alone for local dev, and packagers setup
  BOWER_COMPONENTS_ROOT to '/usr/lib/javascript/' or wherever.
 
 s/lib/share/
 
 However, I'm almost sure that wont be enough to make it work. For
 example, in Debian, we have /usr/share/javascript/angular.__js, not just
 /usr/share/javascript/angular. So django-bower would be searching on the
 wrong path.
 
 
 That is unfortunate. It may be that Debian therefore has to special-case
 angular to handle that case.

I wasn't making a point about Angular here. It's a general issue we have
to take care of addressing correctly.

 I think the general idea of following the component pattern set by bower
 (separate directories with no risk of conflicts, and using the
 bower.json meta-data to allow automatic configuration of the component)
 is too good to dismiss though. Far less work for everyone, including
 packagers.
 
 Perhaps the new packages should have bower in their name?

There's already libjs-angularjs and a bunch of python-xstatic-angular
packages in Debian. I'm not sure that it is necessary to *also* have a
bower-angularjs packages. Why would I need to do that? Just for the
.json file? If that's the case, then couldn't I just add the bower.json
file in the existing libjs-* packages? I'm not sure I get the point here...

Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-20 Thread Richard Jones
On 21 November 2014 16:12, Thomas Goirand z...@debian.org wrote:

 On 11/21/2014 10:52 AM, Richard Jones wrote:
  On 11/18/2014 04:22 PM, Radomir Dopieralski wrote:
   If we use Bower, we don't need to use Xstatic. It would be pure
   overhead. Bower already takes care of tracking releases and
 versions,
   and of bundling the files. All we need is a simple line in the
   settings.py telling Django where it puts all the files -- we don't
   really need Xstatic just for that. The packagers can then take
 those
   Bower packages and turn them into system packages, and just
 add/change
   the paths in settings.py to where they put the files. All in one
   place.
 
  The issue is that there's often not just a single path, but a full
  directory structure to address. That is easily managed with a Debian
  xstatic package, not sure how it would be with Bower.
 
 
  I'm not sure what the difference is (unless it's just related to the
  Debian-specific historical issue you raise below). xstatic and bower are
  remarkably similar a directory to be packaged and some meta-data
  describing it.

 Let me explain again then.

 Let's say there's python-xstatic-foo, and libjs-foo in Debian. If the
 directory structure of libjs-foo is very different from xstatic-foo, I
 can address that issue with symlinks within the xstatic package. Just
 changing the BASE_DIR may not be enough, as libjs-foo may have files
 organized in a very different way than in the upstream package for foo.


OK, so python-xstatic-foo can depend on libjs-foo and just symlink, fair
enough. I'm still not sure what makes bower unique in this respect,
although it'd be nice to avoid creating additional packages just to symlink
things around for bower, which I think is what you're getting at.


 By the way, I went into bower.io http://bower.io as I wanted to
  have a look. How do I
  download a binary package for let's say jasmin? When searching, it
  just links to github...
 
 
  Again; bower is not npm! Jasmine is a command-line program which is
  packaged by npm. Bower packages bundles of stuff that are included in
  web applications. bower itself, a command-line tool, is packaged by npm,
  but itself manages other packages which are not command-line things, but
  just bundles of css, javascript, images, fonts, etc. that are resources
  for web applications to use.

 Sure. But how do I download a bower package then?


I'm not sure I understand the meaning behind this question. bower install
angular downloads a bower package called angular.


 This would only work if upstream package directory structure is the
 same
  as the one in the distribution. For historical reason, that's
  unfortunately often not the case (sometimes because we want to keep
  backward compatibility in the distro because of reverse
 dependencies),
  and just changing the path wont make it work.
 
  On 11/19/2014 03:43 AM, Richard Jones wrote:
   +1 to all that, except I'd recommend using django-bower to handle
 the
   static collection stuff. It's not documented but django-bower has a
   setting BOWER_COMPONENTS_ROOT which would make the above
  transition much
   simpler. You leave it alone for local dev, and packagers setup
   BOWER_COMPONENTS_ROOT to '/usr/lib/javascript/' or wherever.
 
  s/lib/share/
 
  However, I'm almost sure that wont be enough to make it work. For
  example, in Debian, we have /usr/share/javascript/angular.__js, not
 just
  /usr/share/javascript/angular. So django-bower would be searching on
 the
  wrong path.
 
 
  That is unfortunate. It may be that Debian therefore has to special-case
  angular to handle that case.

 I wasn't making a point about Angular here. It's a general issue we have
 to take care of addressing correctly.

  I think the general idea of following the component pattern set by bower
  (separate directories with no risk of conflicts, and using the
  bower.json meta-data to allow automatic configuration of the component)
  is too good to dismiss though. Far less work for everyone, including
  packagers.
 
  Perhaps the new packages should have bower in their name?

 There's already libjs-angularjs and a bunch of python-xstatic-angular
 packages in Debian. I'm not sure that it is necessary to *also* have a
 bower-angularjs packages. Why would I need to do that? Just for the
 .json file? If that's the case, then couldn't I just add the bower.json
 file in the existing libjs-* packages? I'm not sure I get the point here...


The angular component that bower installs looks like this:

$ ls -CAF bower_components/angular
.bower.json angular-csp.css angular.min.js angular.min.js.map package.json
README.md angular.js angular.min.js.gzip bower.json

The bootstrap component looks like this:

$ ls -CAF bower_components/boot/
.bower.json LICENSE bower.json fonts/ js/ package.json
Gruntfile.js README.md dist/ grunt/ 

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-19 Thread Matthias Runge
On 19/11/14 05:25, Richard Jones wrote:
 I've just had a long discussion with #infra folk about the
 global-requirements thing, which deviated (quite naturally) into a
 discussion about packaging (and their thoughts were in line with where
 Radomir and I are heading).
 
 In their view, bower components don't need to be in global-requirements:
 
 - there are no other projects that use bower components, so we don't
 need to ensure cross-project compatibility
 - we can vet new versions of bower components as part of standard
 Horizon change review
 
That sounds good to me! Thanks for doing this!

Matthias


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-19 Thread Fox, Kevin M
Perhaps they are there to support older browsers?

Thanks,
Kevin

From: Matthias Runge [mru...@redhat.com]
Sent: Wednesday, November 19, 2014 12:27 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Horizon] the future of angularjs development in 
Horizon

On 18/11/14 14:48, Thomas Goirand wrote:


 And then, does selenium continues to work for testing Horizon? If so,
 then the solution could be to send the .dll and .xpi files in non-free,
 and remove them from Selenium in main.

Yes, it still works; that leaves the question, why they are included in
the tarball at all.

In Fedora, we do not distribute .dll or selenium xpi files with selenium
at all.

Matthias


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-19 Thread Thomas Goirand
On 11/19/2014 04:27 PM, Matthias Runge wrote:
 On 18/11/14 14:48, Thomas Goirand wrote:
 

 And then, does selenium continues to work for testing Horizon? If so,
 then the solution could be to send the .dll and .xpi files in non-free,
 and remove them from Selenium in main.

 Yes, it still works; that leaves the question, why they are included in
 the tarball at all.
 
 In Fedora, we do not distribute .dll or selenium xpi files with selenium
 at all.
 
 Matthias

Thanks for letting me know. I have opened a bug against the current
selenium package in non-free, to ask to have it uploaded in Debian main,
without the .xpi file. Let's see how it goes.

Cheers,

Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-19 Thread Matthias Runge
On 19/11/14 17:52, Fox, Kevin M wrote:
 Perhaps they are there to support older browsers?
 
Probable.

Windows dlls are quite uncommon in a Linux distribution.

It's a bit unlikely to have an older browser installed in a centrally
managed distribution like Fedora.

Matthias

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-18 Thread Radomir Dopieralski
On 18/11/14 00:59, Richard Jones wrote:
 On 17 November 2014 21:54, Radomir Dopieralski openst...@sheep.art.pl
 mailto:openst...@sheep.art.pl wrote:

 - Bower in the development environment,
 - Bower configuration file in two copies, one for global-requirements,
 and one for the Horizon's local requirements. Plus a gate job that makes
 sure no new library or version gets included in the Horizon's before
 getting into the global-requirements,
 
 Could you perhaps elaborate on this? How do you see the workflow working
 here?

Basically we would have an additional file in the global-requirements
directory, for listing the JavaScript dependencies. Then we would have a
check on the Horizon's gate that would check Horizon's Bower
configuration against that global-requirements file.

This way we keep the same process for JavaScript libraries as we have
for Python libraries: first you submit a patch to the
global-requirements and have the dependency accepted by the infra team
and packagers (checked against licenses, version conflicts, etc.), and
then you add it to Horizon's dependencies. Of course you can submit both
patches at once, just the Horizon one will fail the gate until the
global-requirements one gets merged.

 Given that Horizon already integrates with xstatic, it would be messy
 (and potentially confusing) to include something so it *also* integrated
 with bower. I was envisaging us creating a tool which generates xstatic
 packages from bower packages. I'm not the first to think along these
 lines 
 http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

If we use Bower, we don't need to use Xstatic. It would be pure
overhead. Bower already takes care of tracking releases and versions,
and of bundling the files. All we need is a simple line in the
settings.py telling Django where it puts all the files -- we don't
really need Xstatic just for that. The packagers can then take those
Bower packages and turn them into system packages, and just add/change
the paths in settings.py to where they put the files. All in one place.

 I will be looking into creating such a tool today.

The problem is not the work that has to be done for the initial creation
of the package. That is one-time effort and as you show, it can be
easily automated. The problem is the effort and resources needed to
maintain that package. Someone (the author of the package?) has to check
for security vulnerabilities, critical bugs, packaging issues, changing
licenses, etc. and patch/update the packages accordingly. Also, the more
layers of code you have, them more likely you are to have bugs in the.
Again, I see no reason to duplicate the effort if the Bower packagers
are doing that work for us already (they are, are they?).

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-18 Thread Richard Jones
On 18 November 2014 19:22, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 18/11/14 00:59, Richard Jones wrote:
  On 17 November 2014 21:54, Radomir Dopieralski openst...@sheep.art.pl
  mailto:openst...@sheep.art.pl wrote:
  - Bower in the development environment,
  - Bower configuration file in two copies, one for global-requirements,
  and one for the Horizon's local requirements. Plus a gate job that makes
  sure no new library or version gets included in the Horizon's before
  getting into the global-requirements,
 
  Could you perhaps elaborate on this? How do you see the workflow working
  here?

 Basically we would have an additional file in the global-requirements
 directory, for listing the JavaScript dependencies. Then we would have a
 check on the Horizon's gate that would check Horizon's Bower
 configuration against that global-requirements file.

 This way we keep the same process for JavaScript libraries as we have
 for Python libraries: first you submit a patch to the
 global-requirements and have the dependency accepted by the infra team
 and packagers (checked against licenses, version conflicts, etc.), and
 then you add it to Horizon's dependencies. Of course you can submit both
 patches at once, just the Horizon one will fail the gate until the
 global-requirements one gets merged.

  Given that Horizon already integrates with xstatic, it would be messy
  (and potentially confusing) to include something so it *also* integrated
  with bower. I was envisaging us creating a tool which generates xstatic
  packages from bower packages. I'm not the first to think along these
  lines
 http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

 If we use Bower, we don't need to use Xstatic. It would be pure
 overhead. Bower already takes care of tracking releases and versions,
 and of bundling the files. All we need is a simple line in the
 settings.py telling Django where it puts all the files -- we don't
 really need Xstatic just for that. The packagers can then take those
 Bower packages and turn them into system packages, and just add/change
 the paths in settings.py to where they put the files. All in one place.


I guess I got the message that turning bower packages into system packages
was something that the Linux packagers were not keen on. Did I get the
message wrong there? If so, *and* we can get the bower stuff through #infra
and global-requirements then yes, we should totally try to avoid adding the
xstatic layer :)


Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-18 Thread Matthias Runge
On Tue, Nov 18, 2014 at 09:36:00PM +1100, Richard Jones wrote:
 I guess I got the message that turning bower packages into system packages
 was something that the Linux packagers were not keen on. Did I get the
 message wrong there? If so, *and* we can get the bower stuff through #infra
 and global-requirements then yes, we should totally try to avoid adding the
 xstatic layer :)

For me, it's more work to turn packages into bower, or to translate
bower packages to system packages.

But that is purely because we don't have bower packaged currently in Fedora.
I would vote for the cleaner way (whatever that is).

XStatic is obviously not the cleanest way, but a good compromise in most
situations.

Matthias

-- 
Matthias Runge mru...@redhat.com

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-18 Thread Thomas Goirand
On 11/17/2014 03:22 PM, Matthias Runge wrote:
 On 17/11/14 02:07, Richard Jones wrote:
 Except that selenium is non-free: it's in the non-free repository of
 Debian, because it contains a pre-built .xpi plugin for firefox, which
 itself contains pre-built .so and .dll files.


 Hasn't this issue already been addressed? Horizon currently uses
 Selenium-based integration tests.
 
 For Fedora, we found, that simply removing bundled .dll and .xpi files
 still leaves selenium intact.

And then, does selenium continues to work for testing Horizon? If so,
then the solution could be to send the .dll and .xpi files in non-free,
and remove them from Selenium in main.

Cheers,

Thomas



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-18 Thread Radomir Dopieralski
On 18/11/14 12:54, Matthias Runge wrote:
 On Tue, Nov 18, 2014 at 09:36:00PM +1100, Richard Jones wrote:
 I guess I got the message that turning bower packages into system packages
 was something that the Linux packagers were not keen on. Did I get the
 message wrong there? If so, *and* we can get the bower stuff through #infra
 and global-requirements then yes, we should totally try to avoid adding the
 xstatic layer :)
 
 For me, it's more work to turn packages into bower, or to translate
 bower packages to system packages.
 
 But that is purely because we don't have bower packaged currently in Fedora.
 I would vote for the cleaner way (whatever that is).
 
 XStatic is obviously not the cleanest way, but a good compromise in most
 situations.

The way I thought about it, we would simply replace the Bower packages
with the corresponding system packages, by just changing the path in the
settings.py file.

Maybe an example would help illustrate it.
Currently we have something like this:

STATICFILES_DIRS = [
('horizon/lib/angular',
   xstatic.main.XStatic(xstatic.pkg.angular).base_dir),
('horizon/lib/angular',
   xstatic.main.XStatic(xstatic.pkg.angular_cookies).base_dir),
('horizon/lib/angular',
   xstatic.main.XStatic(xstatic.pkg.angular_mock).base_dir),
('horizon/lib/bootstrap_datepicker',
   xstatic.main.XStatic(xstatic.pkg.bootstrap_datepicker).base_dir),
('bootstrap',
   xstatic.main.XStatic(xstatic.pkg.bootstrap_scss).base_dir),
...
]

We would replace that with:

STATICFILES_DIRS = [
('horizon/lib/angular',
   os.path.join(BASE_DIR, 'bower_modules/angular'),
...
]

or wherever bower puts the files. Now, the packagers would replace those
lines with:

STATICFILES_DIRS = [
('horizon/lib/angular',
   '/usr/lib/javascript/angular'),
...
]

or whatever the packages in their distribution use. It's less work than
having to make a whole xstatic package just to put that single path in
there. The horizon system package would already depend on all the
javascript library packages, so we are sure the files are there.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-18 Thread Richard Jones
On 19 November 2014 03:14, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 18/11/14 12:54, Matthias Runge wrote:
  On Tue, Nov 18, 2014 at 09:36:00PM +1100, Richard Jones wrote:
  I guess I got the message that turning bower packages into system
 packages
  was something that the Linux packagers were not keen on. Did I get the
  message wrong there? If so, *and* we can get the bower stuff through
 #infra
  and global-requirements then yes, we should totally try to avoid adding
 the
  xstatic layer :)
 
  For me, it's more work to turn packages into bower, or to translate
  bower packages to system packages.
 
  But that is purely because we don't have bower packaged currently in
 Fedora.
  I would vote for the cleaner way (whatever that is).
 
  XStatic is obviously not the cleanest way, but a good compromise in most
  situations.

 The way I thought about it, we would simply replace the Bower packages
 with the corresponding system packages, by just changing the path in the
 settings.py file.

 Maybe an example would help illustrate it.
 Currently we have something like this:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular).base_dir),
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular_cookies).base_dir),
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular_mock).base_dir),
 ('horizon/lib/bootstrap_datepicker',
xstatic.main.XStatic(xstatic.pkg.bootstrap_datepicker).base_dir),
 ('bootstrap',
xstatic.main.XStatic(xstatic.pkg.bootstrap_scss).base_dir),
 ...
 ]

 We would replace that with:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
os.path.join(BASE_DIR, 'bower_modules/angular'),
 ...
 ]

 or wherever bower puts the files. Now, the packagers would replace those
 lines with:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
'/usr/lib/javascript/angular'),
 ...
 ]

 or whatever the packages in their distribution use. It's less work than
 having to make a whole xstatic package just to put that single path in
 there. The horizon system package would already depend on all the
 javascript library packages, so we are sure the files are there.
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


+1 to all that, except I'd recommend using django-bower to handle the
static collection stuff. It's not documented but django-bower has a setting
BOWER_COMPONENTS_ROOT which would make the above transition much simpler.
You leave it alone for local dev, and packagers setup BOWER_COMPONENTS_ROOT
to '/usr/lib/javascript/' or wherever.

bower also has a concept of entry points - there a main value in the
bower.json which identifies the Javascript, CSS, font and other files to
include in the index.html to have the library loaded into your application.
Saves a bunch of manual editing to get things right when you include the
library. Sadly, the current django-bower plugin doesn't use any of that,
though it does handle the collectstatic stuff.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-18 Thread Richard Jones
I've just had a long discussion with #infra folk about the
global-requirements thing, which deviated (quite naturally) into a
discussion about packaging (and their thoughts were in line with where
Radomir and I are heading).

In their view, bower components don't need to be in global-requirements:

- there are no other projects that use bower components, so we don't need
to ensure cross-project compatibility
- we can vet new versions of bower components as part of standard Horizon
change review


Richard


On 19 November 2014 06:43, Richard Jones r1chardj0...@gmail.com wrote:

 On 19 November 2014 03:14, Radomir Dopieralski openst...@sheep.art.pl
 wrote:

 On 18/11/14 12:54, Matthias Runge wrote:
  On Tue, Nov 18, 2014 at 09:36:00PM +1100, Richard Jones wrote:
  I guess I got the message that turning bower packages into system
 packages
  was something that the Linux packagers were not keen on. Did I get the
  message wrong there? If so, *and* we can get the bower stuff through
 #infra
  and global-requirements then yes, we should totally try to avoid
 adding the
  xstatic layer :)
 
  For me, it's more work to turn packages into bower, or to translate
  bower packages to system packages.
 
  But that is purely because we don't have bower packaged currently in
 Fedora.
  I would vote for the cleaner way (whatever that is).
 
  XStatic is obviously not the cleanest way, but a good compromise in most
  situations.

 The way I thought about it, we would simply replace the Bower packages
 with the corresponding system packages, by just changing the path in the
 settings.py file.

 Maybe an example would help illustrate it.
 Currently we have something like this:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular).base_dir),
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular_cookies).base_dir),
 ('horizon/lib/angular',
xstatic.main.XStatic(xstatic.pkg.angular_mock).base_dir),
 ('horizon/lib/bootstrap_datepicker',
xstatic.main.XStatic(xstatic.pkg.bootstrap_datepicker).base_dir),
 ('bootstrap',
xstatic.main.XStatic(xstatic.pkg.bootstrap_scss).base_dir),
 ...
 ]

 We would replace that with:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
os.path.join(BASE_DIR, 'bower_modules/angular'),
 ...
 ]

 or wherever bower puts the files. Now, the packagers would replace those
 lines with:

 STATICFILES_DIRS = [
 ('horizon/lib/angular',
'/usr/lib/javascript/angular'),
 ...
 ]

 or whatever the packages in their distribution use. It's less work than
 having to make a whole xstatic package just to put that single path in
 there. The horizon system package would already depend on all the
 javascript library packages, so we are sure the files are there.
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 +1 to all that, except I'd recommend using django-bower to handle the
 static collection stuff. It's not documented but django-bower has a setting
 BOWER_COMPONENTS_ROOT which would make the above transition much simpler.
 You leave it alone for local dev, and packagers setup BOWER_COMPONENTS_ROOT
 to '/usr/lib/javascript/' or wherever.

 bower also has a concept of entry points - there a main value in the
 bower.json which identifies the Javascript, CSS, font and other files to
 include in the index.html to have the library loaded into your application.
 Saves a bunch of manual editing to get things right when you include the
 library. Sadly, the current django-bower plugin doesn't use any of that,
 though it does handle the collectstatic stuff.


  Richard


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-17 Thread Martin Geisler
Thomas Goirand z...@debian.org writes:

Hi Thomas,

 On 11/15/2014 05:34 PM, Martin Geisler wrote:
 I'm sorry if I came across as being hostile towards packagers and
 distros. I've been running Debian for 15 years and that is because of
 the work the Debian developers put into making the system work well
 together at a whole.
 
 When it comes to installing software, I only use apt to touch paths
 outside my home directory. That is to ensure that the integrity of the
 system isn't compromised. That means that software not yet packaged for
 Debian has a low change of being installed by me.
 
 However, the chances of me installing it improve significantly if I can
 install it with pip or npm. Simply because this allows me to do a local
 installation in a home directory -- I know then that I can easily remove
 the sofware later.

 Sorry to say it this way, and it's not about you in particular,

You're quite right, it's not about me! I'm not about to deploy OpenStack
anytime soon so you don't have to sell the packaging solution to me :)

My main goal in this discussion was to bring some web development
knowledge to the table. It's clear to me that you have a very strong
background in Debian packaging -- and (I'm guessing here) not a very
strong background in web development.

 What we care is to find a system that will satisfy both worlds:
 distributions  upstream fast moving development. It is looking like
 NPM has the best feature and that it would be a winner against Bower
 and Grunt.

As Richard said, npm and bower are not competitors. You use npm to
install bower, and you use bower to download Angular, jQuery, Bootstrap
and other static files. These are the static files that you will want to
include when you finally deploy the web app to your server.

Before using Bower, people would simply download Angular from the
projects homepage and check it into version control. Bower is not doing
much, but using it avoids this bad practice.

There is often a kind of compilation step between bower downloading a
dependency and the deployment on the webserver: minification and
compression of the JavaScript and CSS. Concatenating and minifying the
files serve to reduce the number of HTTP requests -- which can make an
app much faster.

Finally, you use Grunt/Gulp to execute other tools during development.
These tools could be a local web server, it could be running the unit
tests. Grunt is only a convenience tool here -- think of it as a kind of
Makefile that tells you how to lunch various tasks.

Again, I'm just trying to bring information to light and let you know
the tools of the trade -- how you and OpenStack as a whole decide to use
and package them is not my concern.

-- 
Martin Geisler

http://google.com/+MartinGeisler


pgpU3qC4oRLtU.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-17 Thread Radomir Dopieralski
On 17/11/14 09:53, Martin Geisler wrote:

[...]

 As Richard said, npm and bower are not competitors. You use npm to
 install bower, and you use bower to download Angular, jQuery, Bootstrap
 and other static files. These are the static files that you will want to
 include when you finally deploy the web app to your server.
 
 Before using Bower, people would simply download Angular from the
 projects homepage and check it into version control. Bower is not doing
 much, but using it avoids this bad practice.
 
 There is often a kind of compilation step between bower downloading a
 dependency and the deployment on the webserver: minification and
 compression of the JavaScript and CSS. Concatenating and minifying the
 files serve to reduce the number of HTTP requests -- which can make an
 app much faster.
 
 Finally, you use Grunt/Gulp to execute other tools during development.
 These tools could be a local web server, it could be running the unit
 tests. Grunt is only a convenience tool here -- think of it as a kind of
 Makefile that tells you how to lunch various tasks.

Thank you for your explanations.

The way I see it, we would need:
- Bower in the development environment,
- Grunt both in the development environment and packaged (to run tests,
etc.),
- Bower configuration file in two copies, one for global-requirements,
and one for the Horizon's local requirements. Plus a gate job that makes
sure no new library or version gets included in the Horizon's before
getting into the global-requirements,
- A tool, probably a script, that would help packaging the Bower
packages into DEB/RPM packages. I suspect the Debian/Fedora packagers
already have a semi-automatic solution for that.
- A script that would generate a file with all the paths to those
packaged libraries, that would get included in Horizon's settings.py

What do you think?
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-17 Thread Richard Jones
On 17 November 2014 21:54, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 17/11/14 09:53, Martin Geisler wrote:

 [...]

  As Richard said, npm and bower are not competitors. You use npm to
  install bower, and you use bower to download Angular, jQuery, Bootstrap
  and other static files. These are the static files that you will want to
  include when you finally deploy the web app to your server.
 
  Before using Bower, people would simply download Angular from the
  projects homepage and check it into version control. Bower is not doing
  much, but using it avoids this bad practice.
 
  There is often a kind of compilation step between bower downloading a
  dependency and the deployment on the webserver: minification and
  compression of the JavaScript and CSS. Concatenating and minifying the
  files serve to reduce the number of HTTP requests -- which can make an
  app much faster.
 
  Finally, you use Grunt/Gulp to execute other tools during development.
  These tools could be a local web server, it could be running the unit
  tests. Grunt is only a convenience tool here -- think of it as a kind of
  Makefile that tells you how to lunch various tasks.

 Thank you for your explanations.

 The way I see it, we would need:
 - Grunt both in the development environment and packaged (to run tests,
 etc.),

I'm increasingly coming to think that we can avoid grunt.

- serve is handled by django runserver
- test running is handled by tox
- livereload could be handled by https://pypi.python.org/pypi/livereload
(it'd be really nice if someone could get support for this rolled in to
django-livereload...)
- watch is not handled by anything and it would be a shame to miss out on
automatic linting/testing, but I think we can live without it

So, the tools we need packaged for Linux are:

- karma
- jasmine (already in Fedora, I believe)
- selenium (I believe this is already done in Fedora and Debian)
- phantomjs (definitely already done!)


 - Bower in the development environment,
 - Bower configuration file in two copies, one for global-requirements,
 and one for the Horizon's local requirements. Plus a gate job that makes
 sure no new library or version gets included in the Horizon's before
 getting into the global-requirements,

Could you perhaps elaborate on this? How do you see the workflow working
here?

Given that Horizon already integrates with xstatic, it would be messy (and
potentially confusing) to include something so it *also* integrated with
bower. I was envisaging us creating a tool which generates xstatic packages
from bower packages. I'm not the first to think along these lines
http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

I will be looking into creating such a tool today.


 - A tool, probably a script, that would help packaging the Bower
 packages into DEB/RPM packages. I suspect the Debian/Fedora packagers
 already have a semi-automatic solution for that.

Yep, that is indeed their problem that they'd have already solved for the
existing xstatic packages.


 - A script that would generate a file with all the paths to those
 packaged libraries, that would get included in Horizon's settings.py

If we stick with xstatic, we don't need this :)


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-17 Thread Richard Jones
On 18 November 2014 10:59, Richard Jones r1chardj0...@gmail.com wrote:

 On 17 November 2014 21:54, Radomir Dopieralski openst...@sheep.art.pl
 wrote:
  - Bower in the development environment,
  - Bower configuration file in two copies, one for global-requirements,
  and one for the Horizon's local requirements. Plus a gate job that makes
  sure no new library or version gets included in the Horizon's before
  getting into the global-requirements,

 Could you perhaps elaborate on this? How do you see the workflow working
 here?

 Given that Horizon already integrates with xstatic, it would be messy (and
 potentially confusing) to include something so it *also* integrated with
 bower. I was envisaging us creating a tool which generates xstatic packages
 from bower packages. I'm not the first to think along these lines
 http://lists.openstack.org/pipermail/openstack-dev/2014-March/031042.html

 I will be looking into creating such a tool today.


I wrote the tool today, and you can find it here:

https://github.com/r1chardj0n3s/flaming-shame

(github auto-named the repository for me - it's like it KNOWS)

I've proposed to Thomas Waldmann that this be included in the xstatic
package.

It doesn't handle dependencies at all - I'm not sure it should or could
sensibly.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-16 Thread Thomas Goirand
On 11/15/2014 06:27 AM, Gabriel Hurley wrote:
 So, I propose that a group gets together and defines criteria:
 we need to accept that the Horizon team (and those knowledgeable
 about web-app development) know best what tools they need, and
 they need to produce such a list as a starting point. We then
 need packagers and maintainers to examine that list and evaluate
 it for problems (non-free software, irresolvable dependencies,
 etc.). They're looking strictly for things which are un-packageable,
 not commenting on the necessity of said software. And we need people
 (thanks, Monty) willing to build new tools to find a way to turn
 that list into something the package maintainers can consume
 without burdening either side.
 
 Make sense?
 
  - Gabriel

I'd be happy to be in this group.

Let me sum-up again my opinion.

Selenium

As I wrote previously, the biggest issue currently for me, is selenium.
It is very frustrating that I can't run these unit tests when building
package, and potentially loose the opportunity to discover problems. I
really would like this to be solved. There's 2 ways of solving it:

1/ Someone works so that the .xpi can be built together with the rest of
selenium, and therefore, selenium can leave the non-free repository of
Debian and go in main (and also be uploaded in other distros).

2/ We move away from Selenium and decide to use something else like
PhantomJS.

Tooling for JS dependencies
===
As for the tooling, we're currently talking about 7 new dependencies,
which isn't much. I believe it's preferable to continue to use XStatic,
because it has been very convenient in many aspects (I wont list here
why again...), and that doing 7 new xstatic packages will not be so much
work. But I wouldn't mind if there was some kind of environment used by
developers to experience new things faster, if that doesn't affect
packaging.

Can someone sum-up the other opinions for the other options?

Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-16 Thread Thomas Goirand
On 11/15/2014 05:34 PM, Martin Geisler wrote:
 I'm sorry if I came across as being hostile towards packagers and
 distros. I've been running Debian for 15 years and that is because of
 the work the Debian developers put into making the system work well
 together at a whole.
 
 When it comes to installing software, I only use apt to touch paths
 outside my home directory. That is to ensure that the integrity of the
 system isn't compromised. That means that software not yet packaged for
 Debian has a low change of being installed by me.
 
 However, the chances of me installing it improve significantly if I can
 install it with pip or npm. Simply because this allows me to do a local
 installation in a home directory -- I know then that I can easily remove
 the sofware later.

Sorry to say it this way, and it's not about you in particular, but this
debate about apt vs others is largely not interesting for the
maintenance of Horizon itself, neither upstream or in distributions.

Let me try to sum-up what has been written, so we can move forward.

What we care is to find a system that will satisfy both worlds:
distributions  upstream fast moving development. It is looking like NPM
has the best feature and that it would be a winner against Bower and Grunt.

At the same time, it's obvious that stuff like NPM cannot be used for
deploying Horizon inside a distribution (please, this is *not* open for
a debate...). Though it seems convenient for development, just like pip
is, because it is very easy to just add a new dependency, and try it out.

Using NPM, Horizon contributors wouldn't have to do the work of building
and maintaining XStatic packages. However, it has the issue that, unlike
with XStatic, these dependencies will *not* land into our
global-requirements.txt, and isn't either integrated (yet) with
something like devstack. We'd have to find a way to clearly define these
dependencies somewhere (like in a global-requirements-js.txt?), and have
a way to agree on them and their versions.

Did I forget anything?

Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-16 Thread Richard Jones
On 17 November 2014 06:42, Thomas Goirand z...@debian.org wrote:

 On 11/15/2014 06:27 AM, Gabriel Hurley wrote:
  So, I propose that a group gets together and defines criteria:
  we need to accept that the Horizon team (and those knowledgeable
  about web-app development) know best what tools they need, and
  they need to produce such a list as a starting point. We then
  need packagers and maintainers to examine that list and evaluate
  it for problems (non-free software, irresolvable dependencies,
  etc.). They're looking strictly for things which are un-packageable,
  not commenting on the necessity of said software. And we need people
  (thanks, Monty) willing to build new tools to find a way to turn
  that list into something the package maintainers can consume
  without burdening either side.
 
  Make sense?
 
   - Gabriel

 I'd be happy to be in this group.

As would I, hence I started the conversation in the first place :)


 Selenium
 
 As I wrote previously, the biggest issue currently for me, is selenium.
 It is very frustrating that I can't run these unit tests when building
 package, and potentially loose the opportunity to discover problems. I
 really would like this to be solved. There's 2 ways of solving it:

 1/ Someone works so that the .xpi can be built together with the rest of
 selenium, and therefore, selenium can leave the non-free repository of
 Debian and go in main (and also be uploaded in other distros).

 2/ We move away from Selenium and decide to use something else like
 PhantomJS.

I think you're confusing a couple of things here. Selenium is a system for
Javascript running tests in a browser environment. To do that, it needs a
browser. PhantomJS provides a headless browser to do that. The tests end up
being faster, less intrusive on a desktop and much simpler to run on
servers (no virtual X11 shenanigans).

I advocate for using PhantomJS, but also for using a reduced Selenium suite
where possible - with an emphasis on unit testing of the angular code
directly. Selenium is just so flaky, especially with an interface that's
even slightly dynamic.


 What we care is to find a system that will satisfy both worlds:
 distributions  upstream fast moving development. It is looking like NPM
 has the best feature and that it would be a winner against Bower and
Grunt.

Again, just to be clear: npm and bower are *both* packaging systems and
have completely separate domains of use. npm is used to package
command-line tools and libraries written in the node.js programming
language whereas bower is used to package browser application components.
It's not npm-or-bower.

bower is most likely going to be replaced by xstatic in our environment,
assuming we have some simple mechanism for creating xstatic packages from
bower components.

The distributions are going to have to package up the npm-packaged tools
that we need, though that set of packages is looking smaller and smaller,
and will probably just include the testing tools (karma, protractor,
jasmin, phantomjs). Some of those are already packaged \o/


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-16 Thread Thomas Goirand
On 11/15/2014 05:03 AM, Matthias Runge wrote:
 On 14/11/14 16:21, Adam Young wrote:
 
 Example:  I don't need Grunt to run a web server.  I need Apache for
 that.  Grunt does not need to be in the distro, mod_wsgi does.
 
 I will need every tool required to run e.g. unit tests or selenium tests
 to be packaged. Why? Because our builders don't have access to the
 internet and I want to be sure, the packaged version of horizon still
 passes tests.
 
 Matthias

Except that selenium is non-free: it's in the non-free repository of
Debian, because it contains a pre-built .xpi plugin for firefox, which
itself contains pre-built .so and .dll files.

So, we're on the same page, but selenium is a bad tool... :)

Chatting with Maxime Vidori on IRC, there's a bunch of alternative
solutions that we could use instead of Selenium. PhantomJS for example
is already in Debian main. I would very much welcome switching to that
instead of Selenium.

Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-16 Thread Richard Jones
On 17 November 2014 11:11, Thomas Goirand z...@debian.org wrote:

 On 11/15/2014 05:03 AM, Matthias Runge wrote:
  On 14/11/14 16:21, Adam Young wrote:
 
  Example:  I don't need Grunt to run a web server.  I need Apache for
  that.  Grunt does not need to be in the distro, mod_wsgi does.
 
  I will need every tool required to run e.g. unit tests or selenium tests
  to be packaged. Why? Because our builders don't have access to the
  internet and I want to be sure, the packaged version of horizon still
  passes tests.
 
  Matthias

 Except that selenium is non-free: it's in the non-free repository of
 Debian, because it contains a pre-built .xpi plugin for firefox, which
 itself contains pre-built .so and .dll files.


Hasn't this issue already been addressed? Horizon currently uses
Selenium-based integration tests.



 So, we're on the same page, but selenium is a bad tool... :)

 Chatting with Maxime Vidori on IRC, there's a bunch of alternative
 solutions that we could use instead of Selenium. PhantomJS for example
 is already in Debian main. I would very much welcome switching to that
 instead of Selenium.


PhantomJS isn't a testing framework - it's just a headless browser that's
scriptable from Javascript. It has a webdriver for Selenium, which is a
testing framework. If you ditch Selenium then you need to replace it with a
testing framework on top of PhantomJS.

There's also CasperJS which is an entirely separate testing framework built
over PhantomJS (and SlimerJS, which looks to be similar to PhantomJS except
built on Gecko - but not headless). It's very JUnit. Some people like that
;)

While Selenium has issues, I'm concerned that there's very few people out
there in the wilds doing interface testing without it. Indeed there's
people advocating for Selenium over PhantomJS directly. Using PhantomJS
directly means we won't have a suite of tests we can then target at IE,
Safari, etc to ensure that the interface works there in an automated
fashion. PhantomJS might be built on WebKit, but it's different enough from
Chrome that you still want to run your tests on Chrome to ensure the
interface actually works. SlimerJS does allow the testing to be slightly
broader, but we'd still miss automated IE and Safari tests.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-16 Thread Matthias Runge
On 17/11/14 02:07, Richard Jones wrote:
 Except that selenium is non-free: it's in the non-free repository of
 Debian, because it contains a pre-built .xpi plugin for firefox, which
 itself contains pre-built .so and .dll files.
 
 
 Hasn't this issue already been addressed? Horizon currently uses
 Selenium-based integration tests.

For Fedora, we found, that simply removing bundled .dll and .xpi files
still leaves selenium intact. But I agree, I would like not to have that
stuff bundled at all.

Tests in Horizon are: unit tests and selenium tests, both executed at
the gate; selenium tests are just mocked, vs. integration tests require
a cloud environment set up. This is not executed at the gate right now.

Matthias

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-16 Thread Radomir Dopieralski
On 15/11/14 03:21, Richard Jones wrote:
 On 15 November 2014 00:58, Radomir Dopieralski openst...@sheep.art.pl wrote:

[...]

 4. additions and upgrades of libraries moderated by the packagers,
 
 Is there already some mechanism for handling the creation and management
 of xstatic packages and how they interact with linux packagers? Sorry if
 this is a noob question.

Anybody can at any time create any Xstatic package and put it on PyPi.
To put it in StackForge, it has to be approved by the infra team, but
they usually don't make problems (also, you don't technically need to
keep the source code on StackForge). To put it in the global
requirements.txt file, it has to be approved by the people who review
that repository, and that includes the packagers. To put it in the
Horizon's requirements.txt, it has to be approved by the Horizon core
reviewers.

I imagine we can have a similar setup for JavaScript dependencies,
possibly a bower configuration file, that would be handled in a similar way.

[...]
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-15 Thread Martin Geisler
Gabriel Hurley gabriel.hur...@nebula.com writes:

Hi Gabriel,

 As the former Horizon PTL, I have a great respect for the importance
 of the contributions the distro maintainers/developers make to Horizon
 and OpenStack as a whole. From how many bugs the distros manage to
 find, to their diligence in vetting the software that we as Horizon
 developers provide, to their overall passion for the work they do.

 It is interesting to me to see the level to which the distros have not
 had to address this problem before. It shows a significant disconnect
 between the Node/JavaScript community and the distros as a whole (not
 surprising since node.js wasn't packaged on many distros until quite
 recently). I'm not excited to see the OpenStack community leading the
 charge on resolving packaging issues that ought to be settled between
 the JS community and the distros. Yet, if we have to in order to move
 forward I would urge us to reach out to the NPM maintainers, major
 library maintainers, etc. to try and make decisions that will benefit
 everyone for years to come.

 It's also interesting to see how strongly people take sides in this
 debate... who is OpenStack for? How crucial are the distros? Obviously
 if you work for RedHat or Canonical the distros are the end-all;
 OpenStack has to be packaged. Other companies? Opinions vary.
 Flexibility on this issue is not consistent, as has been pointed out
 already.

I'm sorry if I came across as being hostile towards packagers and
distros. I've been running Debian for 15 years and that is because of
the work the Debian developers put into making the system work well
together at a whole.

When it comes to installing software, I only use apt to touch paths
outside my home directory. That is to ensure that the integrity of the
system isn't compromised. That means that software not yet packaged for
Debian has a low change of being installed by me.

However, the chances of me installing it improve significantly if I can
install it with pip or npm. Simply because this allows me to do a local
installation in a home directory -- I know then that I can easily remove
the sofware later.

-- 
Martin Geisler

http://google.com/+MartinGeisler


pgpXzFJPPlcp8.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Richard Jones
I think that it boils down to whether it'is possible that distributions:

1. package the node-based tools (grunt, karma, protractor, ...) as
installable programs, and
2. xstatic-package the bower-based packages that we use (probably a couple
of dozen at least).

We might even be able to get away without using grunt, though an
alternative to its LiveReload facility (and one that doesn't then also
depend on another node program like django-livereload does) would be
required. I believe tox and django's runserver (and other manage commands)
could suffice to replace the other functionality typically provided by
grunt.


Richard

On 14 November 2014 18:51, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 13/11/14 23:30, Martin Geisler wrote:

 [...]

  While I agree that it's chaotic, I also think you make the problem worse
  than it really is. First, remember that the user who installs Horizon
  won't need to use the JavaScript based *developer* tools such as npm,
  bower, etc.
 
  That is, I think Horizon developers will use these tools to produce a
  release -- a tarball -- and that tarball will be something you unpack on
  your webserver and then you're done. I base this on what I've seen in
  the project I've been working. The release tarball you download here
  don't mention npm, bower, or any of the other tools:
 
https://github.com/zerovm/swift-browser/releases
 
  The tools were used to produce the tarball and were used to test it, but
  they're not part of the released product. Somewhat similar to how GCC
  isn't included in the tarball if you download a pre-compiled binary.

 [...]

  Maybe a difference is that you don't (yet) install a web application
  like you install a system application. Instead you *deploy* it: you
  unpack files on a webserver, you configure permissions, you setup cache
  rules, you configure a database, etc.

 [...]

 I think I see where the misunderstanding is in this whole discussion. It
 seems it revolves around the purpose and role of the distribution.

 From the naive point of view, the role of a Linux distribution is to
 just collect all the software from respective upstream developers and
 put it in a single repository, so that it can be easily installed by the
 users. That's not the case.

 The role of a distribution is to provide a working ecosystem of
 software, that is:
 a) installed and configured in consistent way,
 b) tested to work with the specific versions that it ships with,
 c) audited for security,
 d) maintained, including security patches,
 e) documented, including external tutorials and the like,
 f) supported, either by the community or by the companies that provide
 support,
 g) free of licensing issues and legal risks as much as possible,
 h) managed with the common system management tools.

 In order to do that, they can't just take a tarball and drop it in a
 directory. They always produce their own builds, to make sure it's the
 same thing that the source code specifies. They sometimes have to
 rearrange configuration files, to make them fit the standards in their
 system. They provide sane configuration defaults. They track the
 security reports about all the installed components, and apply fixes,
 often before the security issue is even announced.

 Basically, a distribution adds a whole bunch of additional guarantees
 for the software they ship. Those are often long-term guarantees, as
 they will be supporting our software long after we have forgotten about
 it already.

 You say that web development doesn't work like that. That may be true,
 but that's mostly because if you develop a new web application in-house,
 and deploy it on your server, you don't really have such large legal
 risk, configuration complexity or support problem -- you just have to
 care about that single application, because the packagers from the
 distribution that you are using are taking care about all the rest of
 software on your server.

 --
 Radomir Dopieralski


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Matthias Runge
On 13/11/14 21:11, Matthew Farina wrote:
 I would like to take a moment to point out that developing system
 software is different from developing web applications. The way systems
 software is developed and often deployed is different from web applications.
 
 Horizon as it sits today appears to be web application development by
 systems software developers. This raises the barrier to entry for web
 application developers.
 
 The approach being proposed moves horizon into the realm of web
 application technologies that web application people use today.
 
 The debate as I'm reading it is about taking web application development
 processes and turning them into systems development processes which are
 often foreign to web application developers. How is this going to work
 out? How will web app people be willing to get involved? Why should this
 be treated the same?
 
 Most of OpenStack is a systems problem. This piece is a little
 different. To make it successful should it get some wiggle room to work
 well in the space it's in?
 
 Note, I'm not saying it should be insecure or anything like that. There
 are just different approaches.
 

Basically, you're saying, we should lower standards to attract more people?

I disagree in your request, to handle horizon different from the rest of
OpenStack: why? And it worked quite well in the past.

This IMHO that's just wrong. When new folks show up: great! Everybody's
welcome.
We might need to educate people here. There are so many patterns used,
and they have been proven wrong in the past.

Technically, it's possible to have several copies of the same library
installed. But because it's possible, that doesn't mean, you should do that.

Matthias




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Matthias Runge
On 13/11/14 19:11, Donald Stufft wrote:

 As far as I’m aware npm supports TLS the same as pip does. That secures the
 transport between the end users and the repository so you can be assured
 that there is no man in the middle. Security wise npm (and pip) are about
 ~95% (mad up numbers, but you can get the gist) of the effectiveness as the
 OS package managers.

Oh, e.g rpm allows packages to be cryptographically signed, and
depending on your systems config, that is enforced. This is quite
different from just tls'ing a connection.

Matthias

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Donald Stufft

 On Nov 14, 2014, at 7:48 AM, Matthias Runge mru...@redhat.com wrote:
 
 On 13/11/14 19:11, Donald Stufft wrote:
 
 As far as I’m aware npm supports TLS the same as pip does. That secures the
 transport between the end users and the repository so you can be assured
 that there is no man in the middle. Security wise npm (and pip) are about
 ~95% (mad up numbers, but you can get the gist) of the effectiveness as the
 OS package managers.
 
 Oh, e.g rpm allows packages to be cryptographically signed, and
 depending on your systems config, that is enforced. This is quite
 different from just tls'ing a connection.

You do realize that TLS provides cryptographic proof of authenticity and
integrity just like PGP does right? (It also provides the cool benefit of
privacy which PGP signing does not).

Generally even with PGP signing you still have a number of online keys sitting
on servers which are able to sign packages and the tooling will accept their
signatures. The essential difference is basically, with TLS you depend on the
web server to not be compromised, with PGP signing you depend on the build
server to not be compromised.

In theory you *can* use PGP signing in a way that all of the signing keys are
offline, however this requires having a person manually sign all artifacts that
are created (and even then, you'd want them to also generate said artifacts
to ensure that they were not compromised). However in the real world, most (if
not all) systems involve online keys.

All this isn't to say that TLS is 100% as good as using something like PGP for
signatures though. PGP does have some good benefits, the major one being that
it travels better/easier/at all. For instance a PGP signature can be
transfered alongside a package file and hosted on untrusted mirrors while
relying on TLS means that you *must* trust the machine from which you're
getting the files from.

TLS is a fairly decent way of securing a package infrastructure though, it
prevents all of the major attacks that PGP signing does in practice but it
moves the high value target from the build machines to the web servers and
makes mirroring require trusting the mirror.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Thomas Goirand
On 11/14/2014 06:30 AM, Martin Geisler wrote:
 That is, I think Horizon developers will use these tools to produce a
 release -- a tarball -- and that tarball will be something you unpack on
 your webserver and then you're done. I base this on what I've seen in
 the project I've been working. The release tarball you download here
 don't mention npm, bower, or any of the other tools:
 
   https://github.com/zerovm/swift-browser/releases
 
 The tools were used to produce the tarball and were used to test it, but
 they're not part of the released product. Somewhat similar to how GCC
 isn't included in the tarball if you download a pre-compiled binary.

When doing packages, I don't even use the tarball, but a git clone,
which itself produces an orig.tar.xz file. I do that to allow more
flexibility and to be able to do upstream code change easily.

So to build a Debian package, I will need to have all the tooling, just
like I need GCC to build packages.

I thought this needed to be cleared out.

On 11/14/2014 06:30 AM, Martin Geisler wrote:
 Maybe a difference is that you don't (yet) install a web application
 like you install a system application. Instead you *deploy* it: you
 unpack files on a webserver, you configure permissions, you setup
 cache rules, you configure a database, etc.

I really don't see why a web application should be any different from
any other component of OpenStack. No, I wont deploy it, I will just
apt-get install it...

On 11/14/2014 06:30 AM, Martin Geisler wrote:
 A web app is something a single user installs on a system (www-data
 or a similar user) and then this user configures the system web
 server to serve this web app.

The configuration part is the role of the package maintainer's script.
At least in Debian, there's the facility to configure apache and https
(if you respond positively to the debconf prompts about this), so
Horizon is directly useable after you install the package. I don't want
this feature to go away.

On 11/14/2014 06:30 AM, Martin Geisler wrote:
 I agree that it would be cool to have web apps be as robust and
 general purpose as system apps. However, I think that day is a ways
 off.

I'm not sure why you are saying this. Horizon works out of the box in
Debian, and so is murano-dashboard and the sahara support.

On 11/14/2014 06:30 AM, Martin Geisler wrote:
 The dependency solver is as good as the community needs it to be. Or
 put differently, if the JavaScript community is able to produce
 working software with npm, then they obviously produce it within the
 bounds of the capabilities of its dependency solver.

 I'm happy to believe that apt has a top-notch and highly tuned
 dependency solver. That doesn't really matter since it would be
 solving problems we don't have.

Dependency solving is pure math. It's very hard to get it right. I don't
agree that some language may need something weaker, and that it's
possible for the maintainers to adapt. It's just that it may, in some
case, be possible to go around some defects if they exist, but everyone
needs a robust dependency solver.

On 11/14/2014 06:30 AM, Martin Geisler wrote:
 In my view, you're taking on way too much work by going into those
 details. I don't think I need or want you do to anything more than
 repack the tarball that npm retrieves -- I don't think you should run
 tests or generate documentation.

Of course, I need to run tests. That's a big part of the QA work, and I
will certainly not give-up on that. You will have a hard time convincing
anyone within the OpenStack community that it's OK to not run unit tests.

As for the doc, well, I do believe it's a big plus.

On 11/14/2014 06:30 AM, Martin Geisler wrote:
 As a user or sysadmin, I would be happy to add a deb line to my
 sources.list and get Debian packages that wrap the node modules.

This means that the packages would *not* be in Debian. Therefore,
horizon couldn't be uploaded to Debian (as there would be some not
available dependencies). That's absolutely not what I want to do. I want
Horizon, just like the rest of OpenStack, to be fully in Debian.

Cheers,

Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Radomir Dopieralski
On 14/11/14 13:02, Richard Jones wrote:
 On 14 November 2014 18:51, Radomir Dopieralski openst...@sheep.art.pl
 mailto:openst...@sheep.art.pl wrote:
 On 13/11/14 23:30, Martin Geisler wrote:
[...]

  Maybe a difference is that you don't (yet) install a web application
  like you install a system application. Instead you *deploy* it: you
  unpack files on a webserver, you configure permissions, you setup cache
  rules, you configure a database, etc.

[...]

 In order to do that, they can't just take a tarball and drop it in a
 directory. They always produce their own builds, to make sure it's the
 same thing that the source code specifies. They sometimes have to
 rearrange configuration files, to make them fit the standards in their
 system. They provide sane configuration defaults. They track the
 security reports about all the installed components, and apply fixes,
 often before the security issue is even announced.

[...]


 I think that it boils down to whether it'is possible that
 distributions:

 1. package the node-based tools (grunt, karma, protractor, ...) as
 installable programs, and
 2. xstatic-package the bower-based packages that we use (probably a
 couple of dozen at least).

 We might even be able to get away without using grunt, though an
 alternative to its LiveReload facility (and one that doesn't then also
 depend on another node program like django-livereload does) would be
 required. I believe tox and django's runserver (and other manage
 commands) could suffice to replace the other functionality typically
 provided by grunt.

We don't really need Xstatic for that. The packagers can as well package
the bower-based packages directly. We can use anything, really,
as long as we follow a process that makes sure that Horizon can be
packaged into the different distributions. That is, we need:

1. All dependencies explicit (with tests failing if a dependency is
   missing),
2. explicit version ranges,
3. no multiple versions of the same library,
4. additions and upgrades of libraries moderated by the packagers,
5. ability to replace the development environment with packaged
   libraries from the system,
6. ability to build and test our software with the tools that can be
   used by all the distributions.

As I said, this is more of a process thing than a tool thing -- I
believe any tool can be used to follow this process, more or less
automatically.

-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Thomas Goirand
On 11/11/2014 03:02 PM, Richard Jones wrote:
 json3
 es5-shim
 angular
 angular-route
 angular-cookies
 angular-animate
 angular-sanitize
 angular-smart-table
 angular-local-storage
 angular-bootstrap
 angular-translate
 font-awesome
 boot
 underscore
 ng-websocket

Just FYI, in Debian, the libjs-angularjs already carries:

angular-route.js
angular-cookies.js
angular-animate.js
angular-sanitize.js

We also already have packaged:
font-awesome
underscore

So, basically, I'd have to package:
json3
es5-shim
boot
angular-smart-table
angular-local-storage
angular-translate
ng-websocket

That's a reasonable amount of work. Multiply this by 2 for the xstatic
packages (if we keep using that), that's about 14 new packages.

By the way, can't we use libjs-sockjs instead of ng-websocket?

Last, I'm ok if we add all these, but please, let's do this in the
beginning of the Kilo cycle. It was really hard to cope with it at the
end of the freeze for Juno.

Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Martin Geisler
Thomas Goirand z...@debian.org writes:

 On 11/14/2014 06:30 AM, Martin Geisler wrote:
 That is, I think Horizon developers will use these tools to produce a
 release -- a tarball -- and that tarball will be something you unpack
 on your webserver and then you're done. I base this on what I've seen
 in the project I've been working. The release tarball you download
 here don't mention npm, bower, or any of the other tools:
 
   https://github.com/zerovm/swift-browser/releases
 
 The tools were used to produce the tarball and were used to test it,
 but they're not part of the released product. Somewhat similar to how
 GCC isn't included in the tarball if you download a pre-compiled
 binary.

 When doing packages, I don't even use the tarball, but a git clone,
 which itself produces an orig.tar.xz file. I do that to allow more
 flexibility and to be able to do upstream code change easily.

That seems to be a choice you're making -- I think you could also use
the upstream tarball as provided (let's say I also include unminified
source files in the tarball).

 On 11/14/2014 06:30 AM, Martin Geisler wrote:
 Maybe a difference is that you don't (yet) install a web application
 like you install a system application. Instead you *deploy* it: you
 unpack files on a webserver, you configure permissions, you setup
 cache rules, you configure a database, etc.

 I really don't see why a web application should be any different from
 any other component of OpenStack. No, I wont deploy it, I will just
 apt-get install it...

I'm a long time Debian user and web developer. I install system tools
(web servers, databases, editors) and I deploy web applications. I
believe that's the most common way to handle web applications today.

 On 11/14/2014 06:30 AM, Martin Geisler wrote:
 I agree that it would be cool to have web apps be as robust and
 general purpose as system apps. However, I think that day is a ways
 off.

 I'm not sure why you are saying this. Horizon works out of the box in
 Debian, and so is murano-dashboard and the sahara support.

That's cool!

 On 11/14/2014 06:30 AM, Martin Geisler wrote:
 The dependency solver is as good as the community needs it to be. Or
 put differently, if the JavaScript community is able to produce
 working software with npm, then they obviously produce it within the
 bounds of the capabilities of its dependency solver.

 I'm happy to believe that apt has a top-notch and highly tuned
 dependency solver. That doesn't really matter since it would be
 solving problems we don't have.

 Dependency solving is pure math. It's very hard to get it right. I don't
 agree that some language may need something weaker, and that it's
 possible for the maintainers to adapt. It's just that it may, in some
 case, be possible to go around some defects if they exist, but everyone
 needs a robust dependency solver.

I think you're misunderstanding the implication. If apt has a stronger
dependency solver than npm, then that's fine. The argument that apt is
stronger than npm is not an argument for moving node packages from npm
to apt -- the Debian packages will still only use a subset of apt
dependency solver, namely the subset they use with npm.

 On 11/14/2014 06:30 AM, Martin Geisler wrote:
 In my view, you're taking on way too much work by going into those
 details. I don't think I need or want you do to anything more than
 repack the tarball that npm retrieves -- I don't think you should run
 tests or generate documentation.

 Of course, I need to run tests. That's a big part of the QA work, and I
 will certainly not give-up on that. You will have a hard time convincing
 anyone within the OpenStack community that it's OK to not run unit tests.

That's not what I said: the OpenStack developers will continue to tests
the software. I personally don't think it's the job of the downstream
packagers to do this QA work. (It's of course cool to run the tests on
the system installed by your packages -- that test run would then
install the needed tools using npm and bower and whatnot if that's how
the upstream has setup the test framework.)

 On 11/14/2014 06:30 AM, Martin Geisler wrote:
 As a user or sysadmin, I would be happy to add a deb line to my
 sources.list and get Debian packages that wrap the node modules.

 This means that the packages would *not* be in Debian. Therefore,
 horizon couldn't be uploaded to Debian (as there would be some not
 available dependencies). That's absolutely not what I want to do. I
 want Horizon, just like the rest of OpenStack, to be fully in Debian.

You don't have to convince me -- I'm not going to deploy OpenStack
anytime soon (apart from DevStack). So I'm not really the right customer
for these packages.

All I wanted to say is that if there is a cool OpenStack dashboard
written as a web app, then I would be happy to deploy it like I deploy
other web apps. If there were a package I could use, that would be cool,
but it's by no means a show-stopper for someone like me.


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Jeremy Stanley
On 2014-11-14 15:10:59 +0100 (+0100), Martin Geisler wrote:
[...]
 That's not what I said: the OpenStack developers will continue to
 tests the software. I personally don't think it's the job of the
 downstream packagers to do this QA work. (It's of course cool to
 run the tests on the system installed by your packages -- that
 test run would then install the needed tools using npm and bower
 and whatnot if that's how the upstream has setup the test
 framework.)
[...]

Just to quibble on this particular point... distro packagers are
also developers. They often (more often than we'd like, and we do
try to find ways to help avoid this where possible) need to carry
their own patches to tweak the software to fit their deployment and
operation model. Being able to rerun tests in-place with packaged
versions of everything including their patches helps them confirm
that what they distribute still works as intended. Further, the
distro users are well within their rights to modify and respin these
packages themselves, and will potentially want to be able to run
these tests for the same reasons.

We distribute our tests as part of our software because our tests
*are* part of our software.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Jeremy Stanley
On 2014-11-14 08:31:37 -0500 (-0500), Donald Stufft wrote:
[...]
 with TLS you depend on the web server to not be compromised
[...]

Or in some cases, the CDN. ;)
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Adam Young

On 11/14/2014 09:05 AM, Thomas Goirand wrote:

That's a reasonable amount of work. Multiply this by 2 for the xstatic
packages (if we keep using that), that's about 14 new packages.

By the way, can't we use libjs-sockjs instead of ng-websocket?

Last, I'm ok if we add all these, but please, let's do this in the
beginning of the Kilo cycle. It was really hard to cope with it at the
end of the freeze for Juno.

Hear hear!  And good work by all of the package maintainers.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Thomas Goirand
On 11/14/2014 08:48 PM, Matthias Runge wrote:
 On 13/11/14 19:11, Donald Stufft wrote:
 
 As far as I’m aware npm supports TLS the same as pip does. That secures the
 transport between the end users and the repository so you can be assured
 that there is no man in the middle. Security wise npm (and pip) are about
 ~95% (mad up numbers, but you can get the gist) of the effectiveness as the
 OS package managers.
 
 Oh, e.g rpm allows packages to be cryptographically signed, and
 depending on your systems config, that is enforced. This is quite
 different from just tls'ing a connection.
 
 Matthias

Just like the Debian Release file is signed into a Release.gpg. So, the
RPM system signs every package, while in Debian, it's the full
repository that is signed. That's 2 different approaches that both
works. pip doesn't offer this kind of security, but at the same time, is
there any kind of check for things that are uploaded to PyPi? Is there
at least a peer review process?

 You do realize that TLS provides cryptographic proof of authenticity
 and integrity just like PGP does right? (It also provides the cool
 benefit of privacy which PGP signing does not).

Do you realize that with the TLS system, you have to trust every and all
CA, while with PGP, you only need to trust a single fingerprint?

 All this isn't to say that TLS is 100% as good as using something
 like PGP for signatures though.

I don't agree. I don't trust the CNNIC or the hong-kong post office,
though their key is on every browser. I do trust the Debian PGP key
generated by the Debian team.

 TLS is a fairly decent way of securing a package infrastructure
 though, it prevents all of the major attacks that PGP signing does in
 practice but it moves the high value target from the build machines
 to the web servers [...]

And ... to a huge list of root CA which you have to trust.

Cheers,

Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Thomas Goirand
On 11/14/2014 09:58 PM, Radomir Dopieralski wrote:
 On 14/11/14 13:02, Richard Jones wrote:
 We might even be able to get away without using grunt, though an
 alternative to its LiveReload facility (and one that doesn't then also
 depend on another node program like django-livereload does) would be
 required. I believe tox and django's runserver (and other manage
 commands) could suffice to replace the other functionality typically
 provided by grunt.
 
 We don't really need Xstatic for that. The packagers can as well package
 the bower-based packages directly. We can use anything, really,
 as long as we follow a process that makes sure that Horizon can be
 packaged into the different distributions. That is, we need:
 
 1. All dependencies explicit (with tests failing if a dependency is
missing),
 2. explicit version ranges,
 3. no multiple versions of the same library,
 4. additions and upgrades of libraries moderated by the packagers,
 5. ability to replace the development environment with packaged
libraries from the system,
 6. ability to build and test our software with the tools that can be
used by all the distributions.

What I liked a lot with the xstatic package thing, is that it was *very*
easy for me to be able to manage path. Horizon just imports the xstatic
package, and then the BASE_DIR or some symlink do the work.

Whatever system we choose, I'd like it to be at least as simple as
xstatic in this regard.

Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Thomas Goirand
On 11/14/2014 10:10 PM, Martin Geisler wrote:
 Of course, I need to run tests. That's a big part of the QA work, and I
  will certainly not give-up on that. You will have a hard time convincing
  anyone within the OpenStack community that it's OK to not run unit tests.
 That's not what I said: the OpenStack developers will continue to tests
 the software. I personally don't think it's the job of the downstream
 packagers to do this QA work. (It's of course cool to run the tests on
 the system installed by your packages -- that test run would then
 install the needed tools using npm and bower and whatnot if that's how
 the upstream has setup the test framework.)

What happens is that the environment within the distribution,
inevitably, will be different from the one ran on the gate. There's
going to be different versions of many components and so on. So it is
very important for me to also run these unit tests, to make sure that
everything continues to work.

Yes, the build-dependencies will pull the same components as pulled by
npm/bower, though they may be installed in different path, and maybe
using different versions.

On 11/14/2014 10:21 PM, Jeremy Stanley wrote: On 2014-11-14 15:10:59
+0100 (+0100), Martin Geisler wrote:
 [...]
 Just to quibble on this particular point... distro packagers are
 also developers. They often (more often than we'd like, and we do
 try to find ways to help avoid this where possible) need to carry
 their own patches to tweak the software to fit their deployment and
 operation model. Being able to rerun tests in-place with packaged
 versions of everything including their patches helps them confirm
 that what they distribute still works as intended. Further, the
 distro users are well within their rights to modify and respin these
 packages themselves, and will potentially want to be able to run
 these tests for the same reasons.

 We distribute our tests as part of our software because our tests
 *are* part of our software.

Exactly! Let me give a few examples...

In Jessie, Nova carries patches so that there is support for Ceph. Until
a few days, there was still an issue with live migration over NFS. This
has just been fixed (thanks to Mehdi!), and running unit tests at build
time confirmed that.

Another example. Jessie will be released with Icehouse Horizon, but with
Django 1.7. The gate didn't test that, but I did. Most patches landed in
Juno, though never Icehouse will be tested with Django 1.7 in the gate.
Lucky, my package runs these unit tests and I can confirm that it
continues to work with Django 1.7 in Jessie.

Hoping these are giving you an insight as why it's *really* important to
run unit tests at build time for distributions,

Cheers,

Thomas Goirand (zigo)

P.S: I also run tempest tests over a Debian package installation to make
sure OpenStack is also functional. But that's another story... :)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Donald Stufft

 On Nov 14, 2014, at 1:57 PM, Thomas Goirand z...@debian.org wrote:
 
 On 11/14/2014 08:48 PM, Matthias Runge wrote:
 On 13/11/14 19:11, Donald Stufft wrote:
 
 As far as I’m aware npm supports TLS the same as pip does. That secures the
 transport between the end users and the repository so you can be assured
 that there is no man in the middle. Security wise npm (and pip) are about
 ~95% (mad up numbers, but you can get the gist) of the effectiveness as the
 OS package managers.
 
 Oh, e.g rpm allows packages to be cryptographically signed, and
 depending on your systems config, that is enforced. This is quite
 different from just tls'ing a connection.
 
 Matthias
 
 Just like the Debian Release file is signed into a Release.gpg. So, the
 RPM system signs every package, while in Debian, it's the full
 repository that is signed. That's 2 different approaches that both
 works. pip doesn't offer this kind of security, but at the same time, is
 there any kind of check for things that are uploaded to PyPi? Is there
 at least a peer review process?

The entirety of PyPI is signed. It’s not possible to get a copy of our 
equivalent
to Release.gpg that isn’t cryptographically proven to have been sent by a server
possessing our RSA private key.

No, PyPI is not a curated repository, nor are any of the language repos that
I’m aware of. That really has nothing to do with securely fetching a particular
package, it only has to do with whether the contents of said package are safe
to use. It means that people installing a package from PyPI have to decide if
they trust the author of the package prior to installing it, but if they do
trust that author then it is roughly as safe to install from PyPI as it is to
install from Debian. The Linux distros are curated repositories so you need to
decide if you want to trust the gatekeepers of the distro instead of the authors
of the software you’re using (or really you probably need to trust both since
a malicious author could likely hide back doors that would go unnoticed during
packaging as a .deb).

 
 You do realize that TLS provides cryptographic proof of authenticity
 and integrity just like PGP does right? (It also provides the cool
 benefit of privacy which PGP signing does not).
 
 Do you realize that with the TLS system, you have to trust every and all
 CA, while with PGP, you only need to trust a single fingerprint?

You absolutely do not need to trust every single CA, or even any CAs at all.
If I recall npm pins which CA they trust. Pip doesn’t (yet) do this because
of some historical reasons but it’s on my list of things as well. It’s no
harder to limit the set of CAs or even individual certificates that are accepted
as valid than it is to limit the set of PGP keys you trust.

 
 All this isn't to say that TLS is 100% as good as using something
 like PGP for signatures though.
 
 I don't agree. I don't trust the CNNIC or the hong-kong post office,
 though their key is on every browser. I do trust the Debian PGP key
 generated by the Debian team.

See above, you’re operating under a misconception that TLS mandates using
the same set of CAs as the browsers use.

 
 TLS is a fairly decent way of securing a package infrastructure
 though, it prevents all of the major attacks that PGP signing does in
 practice but it moves the high value target from the build machines
 to the web servers [...]
 
 And ... to a huge list of root CA which you have to trust.

Already discussed above.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Jeremy Stanley
On 2014-11-15 02:57:15 +0800 (+0800), Thomas Goirand wrote:
[...]
 Do you realize that with the TLS system, you have to trust every
 and all CA, while with PGP, you only need to trust a single
 fingerprint?
[...]

Technically not true *if* the package retrieval tools implement
certificate pinning rather than trusting any old CA (after all,
they're not Web browsers, so they could in theory do that with
minimal impact).

Too bad https://github.com/pypa/pip/issues/1168 hasn't gotten much
traction.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Donald Stufft

 On Nov 14, 2014, at 2:39 PM, Jeremy Stanley fu...@yuggoth.org wrote:
 
 On 2014-11-15 02:57:15 +0800 (+0800), Thomas Goirand wrote:
 [...]
 Do you realize that with the TLS system, you have to trust every
 and all CA, while with PGP, you only need to trust a single
 fingerprint?
 [...]
 
 Technically not true *if* the package retrieval tools implement
 certificate pinning rather than trusting any old CA (after all,
 they're not Web browsers, so they could in theory do that with
 minimal impact).
 
 Too bad https://github.com/pypa/pip/issues/1168 hasn't gotten much
 traction.

Yea, primary reason that hasn’t been done is up until recently we (PyPI)
were relying on the TLS certificate provided by Fastly and they were
unwilling to make a promise to also be using a particular CA for the
next N years. We now have dedicated IP addresses with them so we can
provide them with whatever certificate we want, now it’s just a matter
of selecting CAs and the political process.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Matthias Runge
On 14/11/14 16:21, Adam Young wrote:

 Example:  I don't need Grunt to run a web server.  I need Apache for
 that.  Grunt does not need to be in the distro, mod_wsgi does.

I will need every tool required to run e.g. unit tests or selenium tests
to be packaged. Why? Because our builders don't have access to the
internet and I want to be sure, the packaged version of horizon still
passes tests.

Matthias


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Gabriel Hurley
As the former Horizon PTL, I have a great respect for the importance of the 
contributions the distro maintainers/developers make to Horizon and OpenStack 
as a whole. From how many bugs the distros manage to find, to their diligence 
in vetting the software that we as Horizon developers provide, to their overall 
passion for the work they do.

It is interesting to me to see the level to which the distros have not had to 
address this problem before. It shows a significant disconnect between the 
Node/JavaScript community and the distros as a whole (not surprising since 
node.js wasn't packaged on many distros until quite recently). I'm not excited 
to see the OpenStack community leading the charge on resolving packaging issues 
that ought to be settled between the JS community and the distros. Yet, if we 
have to in order to move forward I would urge us to reach out to the NPM 
maintainers, major library maintainers, etc. to try and make decisions that 
will benefit everyone for years to come.

It's also interesting to see how strongly people take sides in this debate... 
who is OpenStack for? How crucial are the distros? Obviously if you work for 
RedHat or Canonical the distros are the end-all; OpenStack has to be packaged. 
Other companies? Opinions vary. Flexibility on this issue is not consistent, as 
has been pointed out already.

Monty and Adam Young have, I think, voiced a good moderate viewpoint, which is 
that the distros are a core part of OpenStack's success and we have to ensure 
that they can package our software, but that the distros also cannot dictate 
the tools we use in order to produce the best possible product. Distros are 
ultimately middle-men, they provide value to their users and they make sure 
more people use the software we produce. We can all agree that we want to 
maximize the number of people we can reach.

So, I propose that a group gets together and defines criteria: we need to 
accept that the Horizon team (and those knowledgeable about web-app 
development) know best what tools they need, and they need to produce such a 
list as a starting point. We then need packagers and maintainers to examine 
that list and evaluate it for problems (non-free software, irresolvable 
dependencies, etc.). They're looking strictly for things which are 
un-packageable, not commenting on the necessity of said software. And we need 
people (thanks, Monty) willing to build new tools to find a way to turn that 
list into something the package maintainers can consume without burdening 
either side.

Make sense?

 - Gabriel



-Original Message-
From: Thomas Goirand [mailto:z...@debian.org] 
Sent: Friday, November 14, 2014 11:11 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Horizon] the future of angularjs development in 
Horizon

On 11/14/2014 10:10 PM, Martin Geisler wrote:
 Of course, I need to run tests. That's a big part of the QA work, and 
 I
  will certainly not give-up on that. You will have a hard time 
  convincing anyone within the OpenStack community that it's OK to not run 
  unit tests.
 That's not what I said: the OpenStack developers will continue to 
 tests the software. I personally don't think it's the job of the 
 downstream packagers to do this QA work. (It's of course cool to run 
 the tests on the system installed by your packages -- that test run 
 would then install the needed tools using npm and bower and whatnot if 
 that's how the upstream has setup the test framework.)

What happens is that the environment within the distribution, inevitably, will 
be different from the one ran on the gate. There's going to be different 
versions of many components and so on. So it is very important for me to also 
run these unit tests, to make sure that everything continues to work.

Yes, the build-dependencies will pull the same components as pulled by 
npm/bower, though they may be installed in different path, and maybe using 
different versions.

On 11/14/2014 10:21 PM, Jeremy Stanley wrote: On 2014-11-14 15:10:59
+0100 (+0100), Martin Geisler wrote:
 [...]
 Just to quibble on this particular point... distro packagers are also 
 developers. They often (more often than we'd like, and we do try to 
 find ways to help avoid this where possible) need to carry their own 
 patches to tweak the software to fit their deployment and operation 
 model. Being able to rerun tests in-place with packaged versions of 
 everything including their patches helps them confirm that what they 
 distribute still works as intended. Further, the distro users are well 
 within their rights to modify and respin these packages themselves, 
 and will potentially want to be able to run these tests for the same 
 reasons.

 We distribute our tests as part of our software because our tests
 *are* part of our software.

Exactly! Let me give a few examples...

In Jessie, Nova carries patches so that there is support for Ceph. Until a few 
days

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-14 Thread Richard Jones
On 15 November 2014 00:58, Radomir Dopieralski openst...@sheep.art.pl
wrote:

 On 14/11/14 13:02, Richard Jones wrote:
  I think that it boils down to whether it'is possible that
  distributions:
 
  1. package the node-based tools (grunt, karma, protractor, ...) as
  installable programs, and
  2. xstatic-package the bower-based packages that we use (probably a
  couple of dozen at least).
 
  We might even be able to get away without using grunt, though an
  alternative to its LiveReload facility (and one that doesn't then also
  depend on another node program like django-livereload does) would be
  required. I believe tox and django's runserver (and other manage
  commands) could suffice to replace the other functionality typically
  provided by grunt.

 We don't really need Xstatic for that. The packagers can as well package
 the bower-based packages directly. We can use anything, really,
 as long as we follow a process that makes sure that Horizon can be
 packaged into the different distributions. That is, we need:


xstatic provides additional meta-data that makes it much easier to
integrate the static bundle into an application - specifically it
automatically provides the application with file locations and endpoints
needed to be inserted into the application HTML. That stuff would have to
be done manually without xstatic, and would be a configuration pain.



 1. All dependencies explicit (with tests failing if a dependency is
missing),


xstatic provides us with a dependency mechanism through standard Python
setuptools facilities.



 2. explicit version ranges,


xstatic is done through requirements.txt, so yep :)



 3. no multiple versions of the same library,


This is not a thing in bower/client-side land. It's really only an issue
for npm-based packages, and as I've mentioned, the things we're using there
should be packaged as tools by the linux vendor, rather than accessed as
packages by our system. Except on non-Linux, of course, where we'll just
use npm ;)

So I guess the big open question is about how distros are going to deal
with npm tool packaging. I can't really answer that question for them
(except to say: don't try to replace npm's dependency solution with your
own; it simply won't work
because you'll probably never find a set of versions that satisfy even just
the few tools we're looking at for this project).


4. additions and upgrades of libraries moderated by the packagers,


Is there already some mechanism for handling the creation and management of
xstatic packages and how they interact with linux packagers? Sorry if this
is a noob question.



 5. ability to replace the development environment with packaged
libraries from the system,


I would very much like to still use bower for rapid development and
testing, with xstatic coming in once the experimentation is over. But if
there was a tool to automatically generate an xstatic package from a bower
one, that would be eaiser...



 6. ability to build and test our software with the tools that can be
used by all the distributions.


Yep, I think that just implies that the xstatic stuff is done, and that the
distros package the few npm tools we use.


On 15 November 2014 01:05, Thomas Goirand z...@debian.org wrote:

 On 11/11/2014 03:02 PM, Richard Jones wrote:
  json3
  es5-shim
  angular
  angular-route
  angular-cookies
  angular-animate
  angular-sanitize
  angular-smart-table
  angular-local-storage
  angular-bootstrap
  angular-translate
  font-awesome
  boot
  underscore
  ng-websocket

 Just FYI, in Debian, the libjs-angularjs already carries:

 angular-route.js
 angular-cookies.js
 angular-animate.js
 angular-sanitize.js

 We also already have packaged:
 font-awesome
 underscore

 So, basically, I'd have to package:
 json3
 es5-shim
 boot
 angular-smart-table
 angular-local-storage
 angular-translate
 ng-websocket

 That's a reasonable amount of work. Multiply this by 2 for the xstatic
 packages (if we keep using that), that's about 14 new packages.


The issue is integration with the Django application. How do we know what
the file path is to the entry point for that that code, and also how it
differs from other packagers? xstatic takes care of that for us (in much
the same way that bower does), so is valuable.



 By the way, can't we use libjs-sockjs instead of ng-websocket?


They offer different functionality, as far as I can tell. sockjs is a
compatibility layer thing, I think. I'm not sure. The description is a
little vague. On the other hand, ng-websocket is all about providing an
interface for angular applications over the top of websockets (and a handy
mock).



 Last, I'm ok if we add all these, but please, let's do this in the
 beginning of the Kilo cycle. It was really hard to cope with it at the
 end of the freeze for Juno.


I'd imagine this should be reasonable, barring any additional dependencies
we decide to include along the way.

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Martin Geisler
Matthias Runge mru...@redhat.com writes:

 On Wed, Nov 12, 2014 at 08:35:18AM -0500, Monty Taylor wrote:
 Just for the record, I believe that we should chose the tools that
 make sense for making our software, as long as it's not physically
 impossible for them to be packaged. This means we should absolutely
 not use things that require multiple versions of node to be needed.
 The nodejs that's in trusty is new enough to work with all of the
 modern javascript tool chain things needed for this, so other than
 the various javascript tools and libraries not being packaged in the
 distros yet, it should be fine.

 Agreed. We're in the position to describe or define, what we'd like to
 use or to see in the future. That may require us to create required
 tools.

 You're not concerned about node.js? Most probably, since you're not
 distributing it. Looking at the changelog, I'm a bit worried[1]:

 [...]

For better or for worse, the JavaScript community is using Node,
Grunt/Gulp, bower, ... as the default infrastructure tools.

Not using them or putting effort into creating alternatives would be
working against that community and I would say it's wasted effort. That
effort could be put to better use in the core OpenStack code.

I haven't cared much about Node itself, it's just a VM that runs my
JavaScript code. If I were to deploy it on a server I would agree that
the security and robustness becomes critical.

I find npm and bower alright -- they do their job just fine. The
semantic versioning craze is strange to me, but you can avoid it by
fully specifying the versions you depend on.

I find Grunt and Gulp to be overrated. My very simple Gruntfile[1] now
has about 170 lines of JSON to configure the simple tasks. For a copy
this to that task, the JSON format is fine, but for more complex tasks
with several steps it feels forced. I mean, I need to copy files in
several tasks, so I end up with

  copy: {
  some_target: {
  ...
  },
  other_target: {
  ...
  }
  }

in one part of the Gruntfile and then

  task: {
  some_target: {
  ...
  }
  }

many lines away. There's nothing to connect the two pieces of
configuration than a third task that runs both.

The whole multi-task idea also seems strange to me. It feels like an
idea that felt nice when the system was small and now the entire system
is built around it. As an example, running 'grunt copy' by itself is
useless when the two copy targets are small parts of bigger tasks.


About Gulp... I don't get it and I don't buy it :) The implication that
using streams will make your build fast is at best an
over-simplification. While streaming data is cool, elevating this single
idea to the fundamental building block in a simple task runner seems
contrieved.

It forces you into a pattern and you end up writing code looking like:

  gulp.src('./client/templates/*.jade')
.pipe(jade())
.pipe(gulp.dest('./build/templates'))
.pipe(minify())
.pipe(gulp.dest('./build/minified_templates'));

That might look cute for a minimal example like this, but I bet it'll
make things harder than they should be in the future. As in, how can I
inject more files into the stream conditionally? How do I read an
environment variable and inject some extra files? With normal JavaScript
I would know how to do this.

(Here I would apparently use gulp-if together with something called a
lazypipe and I would still need to somehow insert the extra files in the
stream.)

[1]: https://github.com/zerovm/swift-browser/blob/master/Gruntfile.js

-- 
Martin Geisler

http://google.com/+MartinGeisler


pgp8QxY1uE7Ym.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Martin Geisler
Jiri Tomasek jtoma...@redhat.com writes:

 Which tools should we use eventually:

 Based on the contributions by Maxime, Martin and the others, I think
 the list of tools should end up as follows:

 Tooling:
 npm
 bower
 gulp

While I find the design of Gulp strange, I'm sure it will do the job.
Someone said that the Angular teams is moving to it, so that is a +1 in
its favor.

 Jasmine
 Karma/Protractor(?)/eslint

I've used Protractor for my end-to-end tests and after getting to know
it, it works fine. It's my impression that it used to be annying to
getup selenium and actually writing this kind of tests -- with
Protractor you get up and running very quickly.

I don't have anything to compare it with, though, but it's the standard
for Angular development and that alone should be a strong hint.

-- 
Martin Geisler

http://google.com/+MartinGeisler


pgp0giFB1wybi.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Matthias Runge
On 12/11/14 18:23, Jiri Tomasek wrote:

 I see relation between Nodejs and js libs/tools and Angular app defining
 it's dependencies using NPM and Bower quite similar as Ruby, Rubygems
 and Rails application defining it's dependencies in Gemfile.lock.
 Rubygems are being packaged in distros, so why shouldn't node packages?

Some of them are already packaged by distros, and we have even
guidelines to do that:
https://fedoraproject.org/wiki/Packaging:Node.js

But then you'll be using yum/dnf/whatever instead of npm to install it.

Matthias


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Radomir Dopieralski
On 11/11/14 08:02, Richard Jones wrote:

[...]

 There were some discussions around tooling. We're using xstatic to
 manage 3rd party components, but there's a lot missing from that
 environment. I hesitate to add supporting xstatic components on to the
 already large pile of work we have to do, so would recommend we switch
 to managing those components with bower instead. For reference the list
 of 3rd party components I used in angboard* (which is really only a
 teensy fraction of the total application we'd end up with, so this
 components list is probably reduced):

[...]

 Just looking at PyPI, it looks like only a few of those are in xstatic,
 and those are out of date.

There is a very good reason why we only have a few external JavaScript
libraries, and why they are in those versions.

You see, we are not developing Horizon for our own enjoyment, or to
install it at our own webserver and be done with it. What we write has
to be then packaged for different Linux distributions by the packagers.
Those packagers have very little wiggle room with respect to how they
can package it all, and what they can include.

In particular, libraries should get packaged separately, so that they
can upgrade them and apply security patches and so on. Before we used
xstatic, they have to go through the sources of Horizon file by file,
and replace all of our bundled files with symlinks to what is provided
in their distribution. Obviously that was laborious and introduced bugs
when the versions of libraries didn't match.

So now we have the xstatic system. That means, that the libraries are
explicitly listed, with their minimum and maximum version numbers, and
it's easy to make a dummy xstatic package that just points at some
other location of the static files. This simplifies the work of the
packagers.

But the real advantage of using the xstatic packages is that in order to
add them to Horizon, you need to add them to the global-requirements
list, which is being watched and approved by the packagers themselves.
That means, that when you try to introduce a new library, or a version
of an old library, that is for some reason problematic for any of the
distributions (due to licensing issues, due to them needing to remain at
an older version, etc.), they get to veto it and you have a chance of
resolving the problem early, not dropping it at the last moment on the
packagers.

Going back to the versions of the xstatic packages that we use, they are
so old for a reason. Those are the newest versions that are available
with reasonable effort in the distributions for which we make Horizon.

If you want to replace this system with anything else, please keep in
contact with the packagers to make sure that the resulting process makes
sense and is acceptable for them.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Radomir Dopieralski
On 13/11/14 08:23, Matthias Runge wrote:

[...]

 Since we don't require node.js on the server (yet), but only for
 the development process: did anyone look at node's competitors? Like
 CommonJS, Rhino, or SpiderMonkey?

When we were struggling with adding jslint to our CI, we did try a
number of different alternatives to node.js, like Rhino, SpiderMonkey,
V8, phantomjs, etc.

The conclusion was that even tools that advertised themselves as working
on Rhino dropped their support for it several years ago, and just didn't
update the documentation. Node seems to be the only thing that works
without having to modify the code of those tools.

Of course things might have changed since, or we may have someone with
better JavaScript hacking skills who would manage to make it work. But
last year we failed.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Radomir Dopieralski
On 13/11/14 01:32, Richard Jones wrote:
[...]

 We're currently using xstatic and that works with Linux packaging
 because it was designed to cope with being a global installation. The
 current Horizon codebase has a django-xstatic plugin which further makes
 dealing with xstatic components nicer - for example it handles path
 management and static file compilation (JS minification and
 concatenation, for example). That's really nice, but poses some problems:
 
 - we would need to xstatic-ify (and deb/rpm-ify) all those components

Yes. They will need to be deb/rpm/arch/slack/whatever-ified anyways,
because that's how the Linux distributions that are going to ship them work.

 - we could run into global version conflict issues if we run more than
 one service on a system - is this likely to be an issue in practise though?

Yes, this is an issue in practice, and that's why the packagers have a
say in what libraries and in what versions you are adding to the
global-requirements. We have to use versions that are the least problematic.

 - as far as I'm aware, the xstatic JS minification is not angular-aware,
 and will break angular code that has not been written to be
 dumb-minifier-aware (the angular minifier ngMin is written in node and
 knows how to do things more correctly); adding dumb-minifier-awareness
 to angular code makes it ugly and more error-prone :/

You can use any minifier with the django-compress plugin that Horizon
uses (django-xstatic has nothing to do with it). You just define the
command (or a filter written in Python) to use for every mime type.

But I assume that ngMin is written in the Node.js language (which is
superficially similar to JavaScript) and therefore if we used it, you
would have to convince your fellow system administrators to install
node.js on their production servers. Violence may result.

[...]
-- 
Radomir Dopieralski

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Martin Geisler
Radomir Dopieralski openst...@sheep.art.pl writes:

 On 11/11/14 08:02, Richard Jones wrote:

 [...]

 There were some discussions around tooling. We're using xstatic to
 manage 3rd party components, but there's a lot missing from that
 environment. I hesitate to add supporting xstatic components on to
 the already large pile of work we have to do, so would recommend we
 switch to managing those components with bower instead. For reference
 the list of 3rd party components I used in angboard* (which is really
 only a teensy fraction of the total application we'd end up with, so
 this components list is probably reduced):

 [...]

 Just looking at PyPI, it looks like only a few of those are in xstatic,
 and those are out of date.

 There is a very good reason why we only have a few external JavaScript
 libraries, and why they are in those versions.

 You see, we are not developing Horizon for our own enjoyment, or to
 install it at our own webserver and be done with it. What we write has
 to be then packaged for different Linux distributions by the
 packagers. [...]

Maybe a silly question, but why insist on this? Why would you insist on
installing a JavaScript based application using your package manager?

I'm a huge fan of package managers and typically refuse to install
anything globally if it doesn't come as a package.

However, the whole JavaScript ecosystem seems to be centered around the
idea of doing local installations. That means that you no longer need
the package manager to install the software -- you only need a package
manager to install the base system (NodeJs and npm for JavaScript).

Notice that Python has been moving rapidly in the same direction for
years: you only need Python and pip to bootstrap yourself. After getting
used to virtualenv, I've mostly stopped installing Python modules
globally and that is how the JavaScript world expects you to work too.
(Come to think of it, the same applies to some extend to Haskell and
Emacs where there also exist nice package managers that'll pull in and
manage dependencies for you.)

So maybe the Horizon package should be an installer package like the
ones that download fonts or Adobe?

That package would get the right version of node and which then runs the
npm and bower commands to download the rest plus (importantly and much
appreciated) puts the files in a sensible location and gives them good
permissions.

-- 
Martin Geisler

http://google.com/+MartinGeisler


pgph36NhgvYqz.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Thomas Goirand
On 11/13/2014 12:13 PM, Richard Jones wrote:
 the npm stuff is all tool chain; tools
 that I believe should be packaged as such by packagers.

npm is already in Debian:
https://packages.debian.org/sid/npm

However, just like we can't use CPAN, pear install, pip install and
such when building or installing package, we wont be able to use NPM.
This means every single dependency that isn't in Debian will need to be
packaged.

 Horizon is an incredibly complex application. Just so we're all on the
 same page, the components installed by bower for angboard are:
 
 angular
   Because writing an application the size of Horizon without it would be
 madness :)
 angular-route
   Provides structure to the application through URL routing.
 angular-cookies
   Provides management of browser cookies in a way that integrates well
 with angular.
 angular-sanitize
   Allows direct embedding of HTML into angular templates, with sanitization.
 json3
   Compatibility for older browsers so JSON works.
 es5-shim
   Compatibility for older browsers so Javascript (ECMAScript 5) works.
 angular-smart-table
   Table management (population, sorting, filtering, pagination, etc)
 angular-local-storage
Browser local storage with cookie fallback, integrated with angular
 mechanisms.
 angular-bootstrap
Extensions to angular that leverage bootstrap (modal popups, tabbed
 displays, ...)
 font-awesome
Additional glyphs to use in the user interface (warning symbol, info
 symbol, ...)
 boot
Bootstrap for CSS styling (this is the dependency that brings in
 jquery and requirejs)
 underscore
Javascript utility library providing a ton of features Javascript
 lacks but Python programmers expect.
 ng-websocket
Angular-friendly interface to using websockets
 angular-translate
Support for localization in angular using message catalogs generated
 by gettext/transifex.
 angular-mocks
Mocking support for unit testing angular code
 angular-scenario
More support for angular unit tests
 
 Additionally, angboard vendors term.js because it was very poorly
 packaged in the bower ecosystem. +1 for xstatic there I guess :)
 
 So those are the components we needed to create the prototype in a few
 weeks. Not using them would have added months (or possibly years) to the
 development time. Creating an application of the scale of Horizon
 without leveraging all that existing work would be like developing
 OpenStack while barring all use of Python 3rd-party packages.

I have no problem with adding dependencies. That's how things work, for
sure, I just want to make sure it doesn't become hell, with so many
components inter-depending on 100s of them, which would become not
manageable. If we define clear boundaries, then fine! The above seems
reasonable anyway.

Though did you list the dependencies of the above?

Also, if the Horizon project starts using something like NPM (which
again, is already available in Debian, so it has my preference), will we
at least be able to control what version gets in, just like with pip?
Because that's a huge concern for me, and this has been very well and
carefully addressed during the Juno cycle. I would very much appreciate
if the same kind of care was taken again during the Kilo cycle, whatever
path we take. How do I use npm by the way? Any pointer?

Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Thomas Goirand
On 11/13/2014 08:05 PM, Radomir Dopieralski wrote:
 On 11/11/14 08:02, Richard Jones wrote:
 
 [...]
 
 There were some discussions around tooling. We're using xstatic to
 manage 3rd party components, but there's a lot missing from that
 environment. I hesitate to add supporting xstatic components on to the
 already large pile of work we have to do, so would recommend we switch
 to managing those components with bower instead. For reference the list
 of 3rd party components I used in angboard* (which is really only a
 teensy fraction of the total application we'd end up with, so this
 components list is probably reduced):
 
 [...]
 
 Just looking at PyPI, it looks like only a few of those are in xstatic,
 and those are out of date.
 
 There is a very good reason why we only have a few external JavaScript
 libraries, and why they are in those versions.
 
 You see, we are not developing Horizon for our own enjoyment, or to
 install it at our own webserver and be done with it. What we write has
 to be then packaged for different Linux distributions by the packagers.
 Those packagers have very little wiggle room with respect to how they
 can package it all, and what they can include.
 
 In particular, libraries should get packaged separately, so that they
 can upgrade them and apply security patches and so on. Before we used
 xstatic, they have to go through the sources of Horizon file by file,
 and replace all of our bundled files with symlinks to what is provided
 in their distribution. Obviously that was laborious and introduced bugs
 when the versions of libraries didn't match.
 
 So now we have the xstatic system. That means, that the libraries are
 explicitly listed, with their minimum and maximum version numbers, and
 it's easy to make a dummy xstatic package that just points at some
 other location of the static files. This simplifies the work of the
 packagers.
 
 But the real advantage of using the xstatic packages is that in order to
 add them to Horizon, you need to add them to the global-requirements
 list, which is being watched and approved by the packagers themselves.
 That means, that when you try to introduce a new library, or a version
 of an old library, that is for some reason problematic for any of the
 distributions (due to licensing issues, due to them needing to remain at
 an older version, etc.), they get to veto it and you have a chance of
 resolving the problem early, not dropping it at the last moment on the
 packagers.
 
 Going back to the versions of the xstatic packages that we use, they are
 so old for a reason. Those are the newest versions that are available
 with reasonable effort in the distributions for which we make Horizon.
 
 If you want to replace this system with anything else, please keep in
 contact with the packagers to make sure that the resulting process makes
 sense and is acceptable for them.

Thanks a lot for all you wrote above. I 100% agree with it, and you
wrote it better than I would have. Also, I'd like to thank you for the
work we did together during the Juno cycle. Interactions and
communication on IRC were great. I just hope this continues for Kilo, on
the line of what you wrote above.

Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Thomas Goirand
On 11/13/2014 08:32 AM, Richard Jones wrote:
 I note that the Debian JS guidelines* only recommend that libraries
 *should* be minified (though I'm unsure why they even recommend that).

I'm not sure why. Though what *must* be done, is that source packages,
and no point, should ever include a minified version. This should be
done either at build time, or at runtime. There's already some issues
within the current XStatic packages that I had to deal with (eg: remove
these minified versions so it could be uploaded to Debian).

Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Matthias Runge
On 13/11/14 15:56, Martin Geisler wrote:

 Maybe a silly question, but why insist on this? Why would you insist on
 installing a JavaScript based application using your package manager?
 
 I'm a huge fan of package managers and typically refuse to install
 anything globally if it doesn't come as a package.
 
 However, the whole JavaScript ecosystem seems to be centered around the
 idea of doing local installations. That means that you no longer need
 the package manager to install the software -- you only need a package
 manager to install the base system (NodeJs and npm for JavaScript).
Yeah, I understand you.

But: doing local installs or: installing things aside a package manager
means, that software is not maintained, or properly updated any more.
I'm a huge fan of not bundling stuff and re-using libraries from a
central location. Copying foreign code to your own codebase is quite
popular in JavaScript world. That doesn't mean, it's the right thing to do.

Having a package manager pulling updates for your system (rather than
various applications trying to update themselves) is a big feature.

Just try to keep your windows system up to date: how many different
update tools do you need to use? Are you sure, you really got all?
Look at node.js CVEs listed in another mail to this thread. They were
all due to copying foreign code into their code base.

Matthias

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Jiri Tomasek

On 11/13/2014 04:04 PM, Thomas Goirand wrote:

On 11/13/2014 12:13 PM, Richard Jones wrote:

the npm stuff is all tool chain; tools
that I believe should be packaged as such by packagers.

npm is already in Debian:
https://packages.debian.org/sid/npm

However, just like we can't use CPAN, pear install, pip install and
such when building or installing package, we wont be able to use NPM.
This means every single dependency that isn't in Debian will need to be
packaged.


Horizon is an incredibly complex application. Just so we're all on the
same page, the components installed by bower for angboard are:

angular
   Because writing an application the size of Horizon without it would be
madness :)
angular-route
   Provides structure to the application through URL routing.
angular-cookies
   Provides management of browser cookies in a way that integrates well
with angular.
angular-sanitize
   Allows direct embedding of HTML into angular templates, with sanitization.
json3
   Compatibility for older browsers so JSON works.
es5-shim
   Compatibility for older browsers so Javascript (ECMAScript 5) works.
angular-smart-table
   Table management (population, sorting, filtering, pagination, etc)
angular-local-storage
Browser local storage with cookie fallback, integrated with angular
mechanisms.
angular-bootstrap
Extensions to angular that leverage bootstrap (modal popups, tabbed
displays, ...)
font-awesome
Additional glyphs to use in the user interface (warning symbol, info
symbol, ...)
boot
Bootstrap for CSS styling (this is the dependency that brings in
jquery and requirejs)
underscore
Javascript utility library providing a ton of features Javascript
lacks but Python programmers expect.
ng-websocket
Angular-friendly interface to using websockets
angular-translate
Support for localization in angular using message catalogs generated
by gettext/transifex.
angular-mocks
Mocking support for unit testing angular code
angular-scenario
More support for angular unit tests

Additionally, angboard vendors term.js because it was very poorly
packaged in the bower ecosystem. +1 for xstatic there I guess :)

So those are the components we needed to create the prototype in a few
weeks. Not using them would have added months (or possibly years) to the
development time. Creating an application of the scale of Horizon
without leveraging all that existing work would be like developing
OpenStack while barring all use of Python 3rd-party packages.

I have no problem with adding dependencies. That's how things work, for
sure, I just want to make sure it doesn't become hell, with so many
components inter-depending on 100s of them, which would become not
manageable. If we define clear boundaries, then fine! The above seems
reasonable anyway.

Though did you list the dependencies of the above?

Also, if the Horizon project starts using something like NPM (which
again, is already available in Debian, so it has my preference), will we
at least be able to control what version gets in, just like with pip?
Because that's a huge concern for me, and this has been very well and
carefully addressed during the Juno cycle. I would very much appreciate
if the same kind of care was taken again during the Kilo cycle, whatever
path we take. How do I use npm by the way? Any pointer?


NPM and Bower work the similar way as pip, they maintain similar files 
as requirements.txt that list dependencies and it's versions.
I think we should bring up patch that introduces this toolset so we can 
discuss the real amount of dependencies and the process.
It would be also nice to introduce something similar as 
global-requirements.txt in OpenStack project to make sure we have all 
deps in one place and get some approval process on versions used.


Here is an example of random Angular application's package.json (used by 
NPM) and bower.json (used by Bower) files:

http://fpaste.org/150513/89599214/

I'll try to search for a good article that describes how this ecosystem 
works.




Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Jirka

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Martin Geisler
Thomas Goirand z...@debian.org writes:

 Also, if the Horizon project starts using something like NPM (which
 again, is already available in Debian, so it has my preference), will we
 at least be able to control what version gets in, just like with pip?

Yes, npm similarly to pip in that you can specify the versions you want
to install. You can expect loose versions (like 1.2.x if you're okay
with gettting a random patch version) or you can specify the full
version.

In parallel with that, you can add a shrinkwrap file which lists the
versions to install recursively. This locks down the versions of
indirect dependencies too (one of your dependencies might otherwise
depend on a loose version number).

 Because that's a huge concern for me, and this has been very well and
 carefully addressed during the Juno cycle. I would very much appreciate
 if the same kind of care was taken again during the Kilo cycle, whatever
 path we take. How do I use npm by the way? Any pointer?

After installing it, you can try running 'npm install eslint'. That will
create a node_modules folder in your current working directory and
install ESLint inside it. It will also create a cache in ~/.npm.

The ESLint executable is now

  node_modules/.bin/eslint

You'll notice that npm creates

  node_modules/eslint/node_modules/

and install the ESLint dependencies there. Try removing node_modules,
then install one of the dependencies first followed by ESLint:

  rm -r node_modules
  npm install object-assign eslint

This will put both object-assign and eslint at the top of node_modules
and object-assign is no longer in node_modules/eslint/node_modules/.

This works because require('object-assign') in NodeJS will search up the
directory tree until it finds the module. So the ESLint code can still
use object-assign.

You can run 'npm dedupe' to move modules up the tree and de-duplicate
the install somewhat.

This nested module system also works the other way: if you run 'npm
install bower' after installing ESLint, you end up with two versions of
object-assign -- check 'npm list object-assign' for a dependency graph.

Surprisingly and unlike, say, Python, executing

  require('object-assign')

can give you different modules depending on where the code lives that
execute the statement. This allows different parts of Bower to use
different versions of object-assign. This is seen as a feature in this
world... I fear that it can cause strange problems and bugs when data
travels from one part of the program to another.

So, the philosophy behind this is very different from what we're used to
with system-level package managers (focus on local installs) and even
From what we have in the Python world with pip (multiple versions
installed concurrently).

-- 
Martin Geisler

http://google.com/+MartinGeisler


pgp6u4TvkQL2G.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Martin Geisler
Matthias Runge mru...@redhat.com writes:

 On 13/11/14 15:56, Martin Geisler wrote:

 Maybe a silly question, but why insist on this? Why would you insist on
 installing a JavaScript based application using your package manager?
 
 I'm a huge fan of package managers and typically refuse to install
 anything globally if it doesn't come as a package.
 
 However, the whole JavaScript ecosystem seems to be centered around the
 idea of doing local installations. That means that you no longer need
 the package manager to install the software -- you only need a package
 manager to install the base system (NodeJs and npm for JavaScript).
 Yeah, I understand you.

Let me just add that this shift has been a very recent change for me.
With anything but Python and JavaScript, I use my system-level package
manager.

 But: doing local installs or: installing things aside a package
 manager means, that software is not maintained, or properly updated
 any more. I'm a huge fan of not bundling stuff and re-using libraries
 from a central location. Copying foreign code to your own codebase is
 quite popular in JavaScript world. That doesn't mean, it's the right
 thing to do.

I agree that you don't want to copy third-party libraries into your
code. In some sense, that's not what the JavaScript world is doing, at
least not before install time.

What I mean is: the ease of use of local package managers has lead to an
explosion in the number of tiny packages. So JS projects will no longer
copy dependencies into their own project (into their version control
system). They will instead depend on them using a package manager such
as npm or bower.


It seems to me that it should be possible translate the node module into
system level packages in a mechanical fashion, assuming that you're
willing to have a system package for each version of the node module
(you'll need multiple system packages since it's very likely that you'll
end up using multiple different versions at the same time --
alternatively, you could let each system package install every published
or popular node module version).

The guys behind npm has written a little about how that could work here:

  http://nodejs.org/api/modules.html#modules_addenda_package_manager_tips

Has anyone written such wrapper packages? Not the xstatic system which
seems to incur a porting effort -- but really a wrapper system that can
translate any node module into a system package.

-- 
Martin Geisler

http://google.com/+MartinGeisler


pgpvFpbyk_SbN.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Thomas Goirand
On 11/13/2014 10:56 PM, Martin Geisler wrote:
 Maybe a silly question, but why insist on this? Why would you insist on
 installing a JavaScript based application using your package manager?
 
 I'm a huge fan of package managers and typically refuse to install
 anything globally if it doesn't come as a package.
 
 However, the whole JavaScript ecosystem seems to be centered around the
 idea of doing local installations. That means that you no longer need
 the package manager to install the software -- you only need a package
 manager to install the base system (NodeJs and npm for JavaScript).

Yeah... Just like for Java, PHP, Perl, Python, you-name-it...

In what way Javascript will be any different from all of these languages?

 Notice that Python has been moving rapidly in the same direction for
 years: you only need Python and pip to bootstrap yourself. After getting
 used to virtualenv, I've mostly stopped installing Python modules
 globally and that is how the JavaScript world expects you to work too.

Fine for development. Not for deployments. Not for distributions. Or you
just get a huge mess of every library installed 10 times, with 10
different versions, and then a security issue needs to be fixed...

 So maybe the Horizon package should be an installer package like the
 ones that download fonts or Adobe?

This is a horrible design which will *never* make it to distributions.
Please think again. What is it that makes Horizon so special? Answer:
nothing. It's just a web app, so it doesn't need any special care. It
should be packaged, just like the rest of everything, with .deb/.rpm and
so on.

 That package would get the right version of node and which then runs the
 npm and bower commands to download the rest plus (importantly and much
 appreciated) puts the files in a sensible location and gives them good
 permissions.

Fine for your development environment. But that's it.

Also, does your $language-specific-package--manager has enough checks so
that there's no man in the middle attack possible? Is it secured enough?
Can a replay attack be done on it? Does it supports any kind of
cryptography checks like yum or apt does? I'm almost sure that's not the
case. pip is really horrible in this regard. I haven't checked, but I'm
almost sure what we're proposing (eg: npm and such) have the same
weakness. And here, I'm only scratching security concerns. There's other
concerns, like how good is the dependency solver and such (remember: it
took *years* for apt to be as good as it is right now, and it still has
some defects).

On 11/14/2014 12:59 AM, Martin Geisler wrote:
 It seems to me that it should be possible translate the node module
 into system level packages in a mechanical fashion, assuming that
 you're willing to have a system package for each version of the node
 module

Sure! That's how I do most of my Python modules these days. I don't just
create them from scratch, I use my own debpypi script, which generates
a template for packaging. But it can't be fully automated. I could
almost do it in a fully automated manner for PEAR packages for PHP (see
debpear in the Debian archive), but it's harder with Python and pip/PyPi.

Stuff like debian/copyright files have to be processed by hand, and each
package is different (How to run unit tests? nose, testr, pytest? Does
it support python3? Is there a sphinx doc? How good is upstream short
and long description?). I guess it's going to be the same for Javascript
packages: it will be possible to do automation for some parts, but
manual work will always be needed.

On 11/14/2014 12:59 AM, Martin Geisler wrote:
 The guys behind npm has written a little about how that could work
 here:

 http://nodejs.org/api/modules.html#modules_addenda_package_manager_tips

It's fun to read, but very naive. First thing that is very shocking is
that arch independent things gets installed into /usr/lib, where they
belong in /usr/share. If that is what the NPM upstream produces, that's
scary: he doesn't even know how the FSHS (FileSystem Hierarchy Standard)
works.

 Has anyone written such wrapper packages? Not the xstatic system which
 seems to incur a porting effort -- but really a wrapper system that
 can translate any node module into a system package.

The xstatic packages are quite painless, from my view point. What's
painful is to link an existing xstatic package with an already existing
libjs-* package that may have a completely different directory
structure. You can then end-up with a forest of symlinks, but there's no
way around that. No wrapper can solve that problem either. And more
generally, a wrapper that writes a $distribution source package out of a
$language-specific package manager will never solve all, it will only
reduce the amount of packaging work.

Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Donald Stufft

 On Nov 13, 2014, at 12:38 PM, Thomas Goirand z...@debian.org wrote:
 
 On 11/13/2014 10:56 PM, Martin Geisler wrote:
 Maybe a silly question, but why insist on this? Why would you insist on
 installing a JavaScript based application using your package manager?
 
 I'm a huge fan of package managers and typically refuse to install
 anything globally if it doesn't come as a package.
 
 However, the whole JavaScript ecosystem seems to be centered around the
 idea of doing local installations. That means that you no longer need
 the package manager to install the software -- you only need a package
 manager to install the base system (NodeJs and npm for JavaScript).
 
 Yeah... Just like for Java, PHP, Perl, Python, you-name-it...
 
 In what way Javascript will be any different from all of these languages?

Node.js, and npm in particular tends to solve the dependency hell problem
by making it possible to install multiple versions of a particular thing
and use them all within the same process. As far as I know the OS tooling
doesn’t really handle SxS installs of the same thing in multiple versions
very well, I think the closest that you can do is multiple separate packages
with version numbers in the package name? 

In other words it’s entirely possible that a particular set of npm packages
can not be resolved to a single version per library.

 
 Notice that Python has been moving rapidly in the same direction for
 years: you only need Python and pip to bootstrap yourself. After getting
 used to virtualenv, I've mostly stopped installing Python modules
 globally and that is how the JavaScript world expects you to work too.
 
 Fine for development. Not for deployments. Not for distributions. Or you
 just get a huge mess of every library installed 10 times, with 10
 different versions, and then a security issue needs to be fixed…

Eh, I wouldn’t say it’s not fine for deployments. Generally I find that
the less I tie the things where I care about versions to my OS the happier
my life gets. It’s not fine for distributions wanting to offer it though,
that is correct.

 
 So maybe the Horizon package should be an installer package like the
 ones that download fonts or Adobe?
 
 This is a horrible design which will *never* make it to distributions.
 Please think again. What is it that makes Horizon so special? Answer:
 nothing. It's just a web app, so it doesn't need any special care. It
 should be packaged, just like the rest of everything, with .deb/.rpm and
 so on.
 
 That package would get the right version of node and which then runs the
 npm and bower commands to download the rest plus (importantly and much
 appreciated) puts the files in a sensible location and gives them good
 permissions.
 
 Fine for your development environment. But that's it.
 
 Also, does your $language-specific-package--manager has enough checks so
 that there's no man in the middle attack possible? Is it secured enough?
 Can a replay attack be done on it? Does it supports any kind of
 cryptography checks like yum or apt does? I'm almost sure that's not the
 case. pip is really horrible in this regard. I haven't checked, but I'm
 almost sure what we're proposing (eg: npm and such) have the same
 weakness. And here, I'm only scratching security concerns. There's other
 concerns, like how good is the dependency solver and such (remember: it
 took *years* for apt to be as good as it is right now, and it still has
 some defects).

As far as I’m aware npm supports TLS the same as pip does. That secures the
transport between the end users and the repository so you can be assured
that there is no man in the middle. Security wise npm (and pip) are about
~95% (mad up numbers, but you can get the gist) of the effectiveness as the
OS package managers.

 
 On 11/14/2014 12:59 AM, Martin Geisler wrote:
 It seems to me that it should be possible translate the node module
 into system level packages in a mechanical fashion, assuming that
 you're willing to have a system package for each version of the node
 module
 
 Sure! That's how I do most of my Python modules these days. I don't just
 create them from scratch, I use my own debpypi script, which generates
 a template for packaging. But it can't be fully automated. I could
 almost do it in a fully automated manner for PEAR packages for PHP (see
 debpear in the Debian archive), but it's harder with Python and pip/PyPi.

I would be interested to know what makes Python harder in this regard, I
would like to fix it.

 
 Stuff like debian/copyright files have to be processed by hand, and each
 package is different (How to run unit tests? nose, testr, pytest? Does
 it support python3? Is there a sphinx doc? How good is upstream short
 and long description?). I guess it's going to be the same for Javascript
 packages: it will be possible to do automation for some parts, but
 manual work will always be needed.
 
 On 11/14/2014 12:59 AM, Martin Geisler wrote:
 The guys behind npm has written a little about 

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Matthew Farina
I would like to take a moment to point out that developing system software
is different from developing web applications. The way systems software is
developed and often deployed is different from web applications.

Horizon as it sits today appears to be web application development by
systems software developers. This raises the barrier to entry for web
application developers.

The approach being proposed moves horizon into the realm of web application
technologies that web application people use today.

The debate as I'm reading it is about taking web application development
processes and turning them into systems development processes which are
often foreign to web application developers. How is this going to work out?
How will web app people be willing to get involved? Why should this be
treated the same?

Most of OpenStack is a systems problem. This piece is a little different.
To make it successful should it get some wiggle room to work well in the
space it's in?

Note, I'm not saying it should be insecure or anything like that. There are
just different approaches.


On Thu, Nov 13, 2014 at 1:11 PM, Donald Stufft don...@stufft.io wrote:


  On Nov 13, 2014, at 12:38 PM, Thomas Goirand z...@debian.org wrote:
 
  On 11/13/2014 10:56 PM, Martin Geisler wrote:
  Maybe a silly question, but why insist on this? Why would you insist on
  installing a JavaScript based application using your package manager?
 
  I'm a huge fan of package managers and typically refuse to install
  anything globally if it doesn't come as a package.
 
  However, the whole JavaScript ecosystem seems to be centered around the
  idea of doing local installations. That means that you no longer need
  the package manager to install the software -- you only need a package
  manager to install the base system (NodeJs and npm for JavaScript).
 
  Yeah... Just like for Java, PHP, Perl, Python, you-name-it...
 
  In what way Javascript will be any different from all of these languages?

 Node.js, and npm in particular tends to solve the dependency hell problem
 by making it possible to install multiple versions of a particular thing
 and use them all within the same process. As far as I know the OS tooling
 doesn’t really handle SxS installs of the same thing in multiple versions
 very well, I think the closest that you can do is multiple separate
 packages
 with version numbers in the package name?

 In other words it’s entirely possible that a particular set of npm packages
 can not be resolved to a single version per library.

 
  Notice that Python has been moving rapidly in the same direction for
  years: you only need Python and pip to bootstrap yourself. After getting
  used to virtualenv, I've mostly stopped installing Python modules
  globally and that is how the JavaScript world expects you to work too.
 
  Fine for development. Not for deployments. Not for distributions. Or you
  just get a huge mess of every library installed 10 times, with 10
  different versions, and then a security issue needs to be fixed…

 Eh, I wouldn’t say it’s not fine for deployments. Generally I find that
 the less I tie the things where I care about versions to my OS the happier
 my life gets. It’s not fine for distributions wanting to offer it though,
 that is correct.

 
  So maybe the Horizon package should be an installer package like the
  ones that download fonts or Adobe?
 
  This is a horrible design which will *never* make it to distributions.
  Please think again. What is it that makes Horizon so special? Answer:
  nothing. It's just a web app, so it doesn't need any special care. It
  should be packaged, just like the rest of everything, with .deb/.rpm and
  so on.
 
  That package would get the right version of node and which then runs the
  npm and bower commands to download the rest plus (importantly and much
  appreciated) puts the files in a sensible location and gives them good
  permissions.
 
  Fine for your development environment. But that's it.
 
  Also, does your $language-specific-package--manager has enough checks so
  that there's no man in the middle attack possible? Is it secured enough?
  Can a replay attack be done on it? Does it supports any kind of
  cryptography checks like yum or apt does? I'm almost sure that's not the
  case. pip is really horrible in this regard. I haven't checked, but I'm
  almost sure what we're proposing (eg: npm and such) have the same
  weakness. And here, I'm only scratching security concerns. There's other
  concerns, like how good is the dependency solver and such (remember: it
  took *years* for apt to be as good as it is right now, and it still has
  some defects).

 As far as I’m aware npm supports TLS the same as pip does. That secures the
 transport between the end users and the repository so you can be assured
 that there is no man in the middle. Security wise npm (and pip) are about
 ~95% (mad up numbers, but you can get the gist) of the effectiveness as the
 OS package managers.

 

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Thomas Goirand
On 11/14/2014 02:11 AM, Donald Stufft wrote:
 On Nov 13, 2014, at 12:38 PM, Thomas Goirand z...@debian.org wrote:
 On 11/13/2014 10:56 PM, Martin Geisler wrote:
 However, the whole JavaScript ecosystem seems to be centered around the
 idea of doing local installations. That means that you no longer need
 the package manager to install the software -- you only need a package
 manager to install the base system (NodeJs and npm for JavaScript).

 Yeah... Just like for Java, PHP, Perl, Python, you-name-it...

 In what way Javascript will be any different from all of these languages?
 
 Node.js, and npm in particular tends to solve the dependency hell problem
 by making it possible to install multiple versions of a particular thing
 and use them all within the same process. As far as I know the OS tooling
 doesn’t really handle SxS installs of the same thing in multiple versions
 very well, I think the closest that you can do is multiple separate packages
 with version numbers in the package name?

Yeah, and for a very good reason: having multiple version of the same
thing is just really horrible, and should be avoided at all costs.

 Also, does your $language-specific-package--manager has enough checks so
 that there's no man in the middle attack possible? Is it secured enough?
 Can a replay attack be done on it? Does it supports any kind of
 cryptography checks like yum or apt does? I'm almost sure that's not the
 case. pip is really horrible in this regard. I haven't checked, but I'm
 almost sure what we're proposing (eg: npm and such) have the same
 weakness. And here, I'm only scratching security concerns. There's other
 concerns, like how good is the dependency solver and such (remember: it
 took *years* for apt to be as good as it is right now, and it still has
 some defects).
 
 As far as I’m aware npm supports TLS the same as pip does. That secures the
 transport between the end users and the repository so you can be assured
 that there is no man in the middle. Security wise npm (and pip) are about
 ~95% (mad up numbers, but you can get the gist) of the effectiveness as the
 OS package managers.

I don't agree at all with this view. Using TLS is *far* from being
enough IMO. But that's not the point. Using anything else than the
distribution package manager is a hack (or unfinished work).

 On 11/14/2014 12:59 AM, Martin Geisler wrote:
 It seems to me that it should be possible translate the node module
 into system level packages in a mechanical fashion, assuming that
 you're willing to have a system package for each version of the node
 module

 Sure! That's how I do most of my Python modules these days. I don't just
 create them from scratch, I use my own debpypi script, which generates
 a template for packaging. But it can't be fully automated. I could
 almost do it in a fully automated manner for PEAR packages for PHP (see
 debpear in the Debian archive), but it's harder with Python and pip/PyPi.
 
 I would be interested to know what makes Python harder in this regard, I
 would like to fix it.

The fact that the standard from PyPi is very fuzzy is one of the issue.
There's nothing in the format (for example in the DOAP.xml record) that
tells if a module supports Python3 for example. Then the short and long
descriptions aren't respected, often, you get some changelog entries
there. Then there's no real convention for the location of the sphinx
doc. There's also the fact that dependencies for Python have to be
written by hand on a Debian package. See for example, dependencies on
arparse, distribute, ordereddict, which I never put in a Debian package
as it's available in Python 2.7. Or the fact that there's no real unique
place where dependencies are written on a PyPi package (is it hidden
somewhere in setup.py, or is it explicitly written in
requirements.txt?). Etc. On the PHP world, everything is much cleaner,
in the package.xml, which is very easily parse-able.

 On 11/14/2014 12:59 AM, Martin Geisler wrote:
 The guys behind npm has written a little about how that could work
 here:

 http://nodejs.org/api/modules.html#modules_addenda_package_manager_tips

 It's fun to read, but very naive. First thing that is very shocking is
 that arch independent things gets installed into /usr/lib, where they
 belong in /usr/share. If that is what the NPM upstream produces, that's
 scary: he doesn't even know how the FSHS (FileSystem Hierarchy Standard)
 works.
 
 I may be wrong, but doesn’t the FHS state that /usr/share is for arch
 independent *data* that is read only?

No, that's for arch independent *things*. Like for example, javascript.
In Debian, these are going in /usr/share/javascript. Python code used to
live within /usr/share/pyshared too (but we stopped the symlink forest
during the Jessie cycle).

 I believe it also states that
 /usr/lib is for object files, libraries, and internal binaries.

It's for arch dependent things.

 As far as
 I’m aware the things that npm installs are libraries the same 

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Martin Geisler
Thomas Goirand z...@debian.org writes:

 On 11/13/2014 10:56 PM, Martin Geisler wrote:
 Maybe a silly question, but why insist on this? Why would you insist on
 installing a JavaScript based application using your package manager?
 
 I'm a huge fan of package managers and typically refuse to install
 anything globally if it doesn't come as a package.
 
 However, the whole JavaScript ecosystem seems to be centered around the
 idea of doing local installations. That means that you no longer need
 the package manager to install the software -- you only need a package
 manager to install the base system (NodeJs and npm for JavaScript).

 Yeah... Just like for Java, PHP, Perl, Python, you-name-it...

 In what way Javascript will be any different from all of these
 languages?

Let me again say that I'm fairly new to this modern JavaScript world. I
knew almost nothing about node, npm, bower, and grunt six months ago.

One answer may be that there isn't an expectation in the JavaScript
community that you'll be reusing system libraries the same way that
there is in at least the Python and Perl communities.

It's my impression that the JavaScript world is used to move *very*
quickly. People release versions very rapidly and are happy to break
APIs. I think they're okay with it because of semver -- the idea that as
long as you increment the right digit in your version number, you don't
have to care (much) about the work you put on your users when you ask
them to upgrade.

I hope that's too harsh a description, but the way the node module
system is explicitly designed to allow a single running program to use
multiple versions of the *same* module hints that this chaotic situation
is both expected and considered normal.

When reading about the module system, I came across blog posts that
called it superior compared to, say, Python, exactly because of this
flexibility. Reasonable people are sure to disagree, but that seems to
be the current situation.

 Notice that Python has been moving rapidly in the same direction for
 years: you only need Python and pip to bootstrap yourself. After getting
 used to virtualenv, I've mostly stopped installing Python modules
 globally and that is how the JavaScript world expects you to work too.

 Fine for development. Not for deployments. Not for distributions. Or you
 just get a huge mess of every library installed 10 times, with 10
 different versions, and then a security issue needs to be fixed...

While I agree that it's chaotic, I also think you make the problem worse
than it really is. First, remember that the user who installs Horizon
won't need to use the JavaScript based *developer* tools such as npm,
bower, etc.

That is, I think Horizon developers will use these tools to produce a
release -- a tarball -- and that tarball will be something you unpack on
your webserver and then you're done. I base this on what I've seen in
the project I've been working. The release tarball you download here
don't mention npm, bower, or any of the other tools:

  https://github.com/zerovm/swift-browser/releases

The tools were used to produce the tarball and were used to test it, but
they're not part of the released product. Somewhat similar to how GCC
isn't included in the tarball if you download a pre-compiled binary.

 So maybe the Horizon package should be an installer package like the
 ones that download fonts or Adobe?

 This is a horrible design which will *never* make it to distributions.
 Please think again. What is it that makes Horizon so special? Answer:
 nothing. It's just a web app, so it doesn't need any special care.
 It should be packaged, just like the rest of everything, with
 .deb/.rpm and so on.

Maybe a difference is that you don't (yet) install a web application
like you install a system application. Instead you *deploy* it: you
unpack files on a webserver, you configure permissions, you setup cache
rules, you configure a database, etc.

I find this to be quite different from, say, installing Emacs. Emacs is
something you install once on a system and this single installation can
be done in a right way so that it's useable for several users on the
system.

A web app is something a single user installs on a system (www-data or a
similar user) and then this user configures the system web server to
serve this web app.

I agree that it would be cool to have web apps be as robust and general
purpose as system apps. However, I think that day is a ways off.

 That package would get the right version of node and which then runs
 the npm and bower commands to download the rest plus (importantly and
 much appreciated) puts the files in a sensible location and gives
 them good permissions.

 Fine for your development environment. But that's it.

 Also, does your $language-specific-package--manager has enough checks
 so that there's no man in the middle attack possible? Is it secured
 enough? Can a replay attack be done on it? Does it supports any kind
 of cryptography checks like 

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Donald Stufft

 On Nov 13, 2014, at 5:23 PM, Thomas Goirand z...@debian.org wrote:
 
 On 11/14/2014 02:11 AM, Donald Stufft wrote:
 On Nov 13, 2014, at 12:38 PM, Thomas Goirand z...@debian.org wrote:
 On 11/13/2014 10:56 PM, Martin Geisler wrote:
 However, the whole JavaScript ecosystem seems to be centered around the
 idea of doing local installations. That means that you no longer need
 the package manager to install the software -- you only need a package
 manager to install the base system (NodeJs and npm for JavaScript).
 
 Yeah... Just like for Java, PHP, Perl, Python, you-name-it...
 
 In what way Javascript will be any different from all of these languages?
 
 Node.js, and npm in particular tends to solve the dependency hell problem
 by making it possible to install multiple versions of a particular thing
 and use them all within the same process. As far as I know the OS tooling
 doesn’t really handle SxS installs of the same thing in multiple versions
 very well, I think the closest that you can do is multiple separate packages
 with version numbers in the package name?
 
 Yeah, and for a very good reason: having multiple version of the same
 thing is just really horrible, and should be avoided at all costs.

I don’t disagree with you that I don’t particularly like that situation, just
saying that node.js/npm *is* special in this regard because it’s entirely
possible that you can’t resolve things to a single version per dependency and
their tooling will just work for that.

 
 Also, does your $language-specific-package--manager has enough checks so
 that there's no man in the middle attack possible? Is it secured enough?
 Can a replay attack be done on it? Does it supports any kind of
 cryptography checks like yum or apt does? I'm almost sure that's not the
 case. pip is really horrible in this regard. I haven't checked, but I'm
 almost sure what we're proposing (eg: npm and such) have the same
 weakness. And here, I'm only scratching security concerns. There's other
 concerns, like how good is the dependency solver and such (remember: it
 took *years* for apt to be as good as it is right now, and it still has
 some defects).
 
 As far as I’m aware npm supports TLS the same as pip does. That secures the
 transport between the end users and the repository so you can be assured
 that there is no man in the middle. Security wise npm (and pip) are about
 ~95% (mad up numbers, but you can get the gist) of the effectiveness as the
 OS package managers.
 
 I don't agree at all with this view. Using TLS is *far* from being
 enough IMO. But that's not the point. Using anything else than the
 distribution package manager is a hack (or unfinished work).

This is an argument that I don’t think either of us will convince the other of,
so I’ll just say agree to disagree.

 
 On 11/14/2014 12:59 AM, Martin Geisler wrote:
 It seems to me that it should be possible translate the node module
 into system level packages in a mechanical fashion, assuming that
 you're willing to have a system package for each version of the node
 module
 
 Sure! That's how I do most of my Python modules these days. I don't just
 create them from scratch, I use my own debpypi script, which generates
 a template for packaging. But it can't be fully automated. I could
 almost do it in a fully automated manner for PEAR packages for PHP (see
 debpear in the Debian archive), but it's harder with Python and pip/PyPi.
 
 I would be interested to know what makes Python harder in this regard, I
 would like to fix it.
 
 The fact that the standard from PyPi is very fuzzy is one of the issue.
 There's nothing in the format (for example in the DOAP.xml record) that
 tells if a module supports Python3 for example. Then the short and long
 descriptions aren't respected, often, you get some changelog entries
 there. Then there's no real convention for the location of the sphinx
 doc. There's also the fact that dependencies for Python have to be
 written by hand on a Debian package. See for example, dependencies on
 arparse, distribute, ordereddict, which I never put in a Debian package
 as it's available in Python 2.7. Or the fact that there's no real unique
 place where dependencies are written on a PyPi package (is it hidden
 somewhere in setup.py, or is it explicitly written in
 requirements.txt?). Etc. On the PHP world, everything is much cleaner,
 in the package.xml, which is very easily parse-able.

(This is fairly off topic, so if you want to reply to this in private that’s
fine):

Nothing that says if it supports py3:
Yea, this is a problem, you can somewhat estimate it using the Python 3
classifier though.

Short and Long descriptions aren’t respected:
I’m not sure what you mean by isn’t respected?

No real convention for the location of the sphinx docs:
Ok, I’ll add this to the list of things that needs work.

Have to write dependencies by hand:
Not sure what you mean by not depending on argparse, distribute, 
ordereddict,
etc? 

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Richard Jones
On 14 November 2014 02:04, Thomas Goirand z...@debian.org wrote:

 On 11/13/2014 12:13 PM, Richard Jones wrote:
  the npm stuff is all tool chain; tools
  that I believe should be packaged as such by packagers.

 npm is already in Debian:
 https://packages.debian.org/sid/npm

 However, just like we can't use CPAN, pear install, pip install and
 such when building or installing package, we wont be able to use NPM.
 This means every single dependency that isn't in Debian will need to be
 packaged.


Just to be clearer, when I wrote the npm stuff I meant npm and the tools
installed by it, so grunt, bower, karma, phantomjs, etc. Not the stuff
managed by bower, just the stuff installed by npm. Those npm-based things
should be packaged by the distros as tools, just like other programs the
distros package.


 Horizon is an incredibly complex application. Just so we're all on the
  same page, the components installed by bower for angboard are:
 
  angular
Because writing an application the size of Horizon without it would be
  madness :)
  angular-route
Provides structure to the application through URL routing.
  angular-cookies
Provides management of browser cookies in a way that integrates well
  with angular.
  angular-sanitize
Allows direct embedding of HTML into angular templates, with
 sanitization.
  json3
Compatibility for older browsers so JSON works.
  es5-shim
Compatibility for older browsers so Javascript (ECMAScript 5) works.
  angular-smart-table
Table management (population, sorting, filtering, pagination, etc)
  angular-local-storage
 Browser local storage with cookie fallback, integrated with angular
  mechanisms.
  angular-bootstrap
 Extensions to angular that leverage bootstrap (modal popups, tabbed
  displays, ...)
  font-awesome
 Additional glyphs to use in the user interface (warning symbol, info
  symbol, ...)
  boot
 Bootstrap for CSS styling (this is the dependency that brings in
  jquery and requirejs)
  underscore
 Javascript utility library providing a ton of features Javascript
  lacks but Python programmers expect.
  ng-websocket
 Angular-friendly interface to using websockets
  angular-translate
 Support for localization in angular using message catalogs generated
  by gettext/transifex.
  angular-mocks
 Mocking support for unit testing angular code
  angular-scenario
 More support for angular unit tests
 
  Additionally, angboard vendors term.js because it was very poorly
  packaged in the bower ecosystem. +1 for xstatic there I guess :)
 
  So those are the components we needed to create the prototype in a few
  weeks. Not using them would have added months (or possibly years) to the
  development time. Creating an application of the scale of Horizon
  without leveraging all that existing work would be like developing
  OpenStack while barring all use of Python 3rd-party packages.

 I have no problem with adding dependencies. That's how things work, for
 sure, I just want to make sure it doesn't become hell, with so many
 components inter-depending on 100s of them, which would become not
 manageable. If we define clear boundaries, then fine! The above seems
 reasonable anyway.

 Though did you list the dependencies of the above?


Again, just so we're clear, yes, the above is *all* the components
installed by bower, including dependencies (jquery and requirejs being the
*only* dependencies not directly installed).

As I mentioned, the dependency trees in bower are significantly simpler
than npm packages tend to be (most bower packages have zero or one
dependency). The 100s of dependencies are in npm packages - but as Martin
Gleiser has pointed out, npm solves that problem with its local install and
node_modules directory structures.


 Richard
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-13 Thread Radomir Dopieralski
On 13/11/14 23:30, Martin Geisler wrote:

[...]

 While I agree that it's chaotic, I also think you make the problem worse
 than it really is. First, remember that the user who installs Horizon
 won't need to use the JavaScript based *developer* tools such as npm,
 bower, etc.
 
 That is, I think Horizon developers will use these tools to produce a
 release -- a tarball -- and that tarball will be something you unpack on
 your webserver and then you're done. I base this on what I've seen in
 the project I've been working. The release tarball you download here
 don't mention npm, bower, or any of the other tools:
 
   https://github.com/zerovm/swift-browser/releases
 
 The tools were used to produce the tarball and were used to test it, but
 they're not part of the released product. Somewhat similar to how GCC
 isn't included in the tarball if you download a pre-compiled binary.

[...]

 Maybe a difference is that you don't (yet) install a web application
 like you install a system application. Instead you *deploy* it: you
 unpack files on a webserver, you configure permissions, you setup cache
 rules, you configure a database, etc.

[...]

I think I see where the misunderstanding is in this whole discussion. It
seems it revolves around the purpose and role of the distribution.

From the naive point of view, the role of a Linux distribution is to
just collect all the software from respective upstream developers and
put it in a single repository, so that it can be easily installed by the
users. That's not the case.

The role of a distribution is to provide a working ecosystem of
software, that is:
a) installed and configured in consistent way,
b) tested to work with the specific versions that it ships with,
c) audited for security,
d) maintained, including security patches,
e) documented, including external tutorials and the like,
f) supported, either by the community or by the companies that provide
support,
g) free of licensing issues and legal risks as much as possible,
h) managed with the common system management tools.

In order to do that, they can't just take a tarball and drop it in a
directory. They always produce their own builds, to make sure it's the
same thing that the source code specifies. They sometimes have to
rearrange configuration files, to make them fit the standards in their
system. They provide sane configuration defaults. They track the
security reports about all the installed components, and apply fixes,
often before the security issue is even announced.

Basically, a distribution adds a whole bunch of additional guarantees
for the software they ship. Those are often long-term guarantees, as
they will be supporting our software long after we have forgotten about
it already.

You say that web development doesn't work like that. That may be true,
but that's mostly because if you develop a new web application in-house,
and deploy it on your server, you don't really have such large legal
risk, configuration complexity or support problem -- you just have to
care about that single application, because the packagers from the
distribution that you are using are taking care about all the rest of
software on your server.

-- 
Radomir Dopieralski


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Matthias Runge
On 12/11/14 08:40, Richard Jones wrote:

 
 I believe the nodeenv method of installing node solves this, as it's
 entirely local to the development environment.

See below, this touches package build as well.
 
 
 I will have to go through all dependencies and do a review, if those are
 acceptable for inclusion e.g in Fedora. The same is true for Thomas
 Goirand for inclusion in Debian.
 
 
  Petr Belanyi has added optional jshint install for js linting into
  Horizon and it installs nodejs as it depends on it. Could this approach
  work for our need of js tooling too? [1]
 
 Sigh, this nonsense doesn't go away? This is the third time the same
 issue comes up.
 
 jshint is NOT free software.
 
 https://github.com/jshint/jshint/blob/master/src/jshint.js#L19 
 
 
 They're trying to resolve that https://github.com/jshint/jshint/issues/1234
 
 But regardless, jshint doesn't have to be installed from a Linux
 repository; it's usually installed using npm alongside the other node tools.
 
Thanks for the pointer, this is good news!

Regarding package managers, my POV is completely different. From a
distributor perspective, where customers expect everything provided from
a single source, I'm not using npm, pip etc. I'm packaging that stuff
properly before using it.

There are companies out there, where security policy does not allow to
install software from a third party repository. pypi etc. are considered
as a third party here in this context.

I would prefer to have the complete tool chain available as (RPM)
packages. I am executing unit tests etc. during package build. Our
builders don't have access to the internet, downloading any other stuff
from the internet is no option.

Matthias





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Matthias Runge
On 11/11/14 08:02, Richard Jones wrote:

 There were some discussions around tooling. We're using xstatic to
 manage 3rd party components, but there's a lot missing from that
 environment. I hesitate to add supporting xstatic components on to the
 already large pile of work we have to do, so would recommend we switch
 to managing those components with bower instead. For reference the list
 of 3rd party components I used in angboard* (which is really only a
 teensy fraction of the total application we'd end up with, so this
 components list is probably reduced):
 
 json3
 es5-shim
 angular
 angular-route
 angular-cookies
 angular-animate
 angular-sanitize
 angular-smart-table
 angular-local-storage
 angular-bootstrap
 angular-translate
 font-awesome
 boot
 underscore
 ng-websocket
 
 Just looking at PyPI, it looks like only a few of those are in xstatic,
 and those are out of date.

Richard,

thank you for writing this up!

bower is (not yet) available e.g in Fedora, Debian, Ubuntu.

I'm quite a bit skeptical why the need for 3 additional package managers
(npm, grunt, bower) for simply installing javascript files.

Looking at es5-shim, it pulls in additional 28 dependent packages, json3
has 12 dependencies (including a circular dependency, one circular
depencency in dependencies),

Not counting dependencies in dependent packages, I can only suspect,
this will bring us at least 100 new packages required.

Once this is finalized, we'll need people to walk through all deps and
put packages up for review, to have those available when kilo is ready.

Matthias


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Matthias Runge
On 12/11/14 09:28, Matthias Runge wrote:

 
 Looking at es5-shim, it pulls in additional 28 dependent packages, json3
 has 12 dependencies (including a circular dependency, one circular
 depencency in dependencies),
Please scratch that. I'll need to look at that a bit deeper (after
another coffee)

Matthias


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Martin Geisler
Richard Jones r1chardj0...@gmail.com writes:

 On 12 November 2014 18:17, Matthias Runge mru...@redhat.com wrote:

 Sigh, this nonsense doesn't go away? This is the third time the same
 issue comes up.

 jshint is NOT free software.

 https://github.com/jshint/jshint/blob/master/src/jshint.js#L19


 They're trying to resolve that https://github.com/jshint/jshint/issues/1234

I've switched to ESLint for my own projects:

  http://eslint.org/

I find it to be better documented and easier to extend than JSHint. Plus
it has had a sane license from the beginning :)

-- 
Martin Geisler

http://google.com/+MartinGeisler


pgpwAiqU4ITyx.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Maxime Vidori
Hello all,

I will write here some things I have noticed in the mailing list and on which I 
can contribute.

Npm and Bower: 
bower and npm are here for two separate things, the first one is here to handle 
the dependencies of the webapp (angular, es5shim, d3.js and so on), meanwhile 
npm is here to handle the development environment (karma, grunt, jasmine, 
etc...). There is a possibility to avoid the use of bower by using npm alone, 
but I never tested that so a POC can be required.

Grunt vs Gulp:
Both are equivalent but I have a preference for the second one. It is more 
powerful than grunt, more customizable, easier to understand for new comers. 
All the plugins for an angular development exists and are quite stable. Last 
but not least the angular team is now pushing to replace its infra from grunt 
to gulp. But like I said both are equivalent and can be switched easily, so if 
you are more comfortable with grunt it is not a big deal.

Tastipy vs Djangular:
Like it has already been said we will use a restful API to handle the client 
server exchanges, so Tastipy seems to be preferable over Djangular.

Jasmine vs Mocha + Chai:
I am not really aware of Mocha + Chai features, but I have already used Jasmine 
for all my angular testing and it feeds all my needs, please use the version 
2.0 of it for its powerful features on asynchronous testing and the clean 
handle of variables in the testings. Last it is only one library which already 
have plugins for the karma test runner.

I do not know if I addressed all the questions but here is a starting.

Cheers 

Max


- Original Message -
From: Matthias Runge mru...@redhat.com
To: openstack-dev@lists.openstack.org
Sent: Wednesday, November 12, 2014 9:39:29 AM
Subject: Re: [openstack-dev] [Horizon] the future of angularjs development in 
Horizon

On 12/11/14 09:28, Matthias Runge wrote:

 
 Looking at es5-shim, it pulls in additional 28 dependent packages, json3
 has 12 dependencies (including a circular dependency, one circular
 depencency in dependencies),
Please scratch that. I'll need to look at that a bit deeper (after
another coffee)

Matthias


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Martin Geisler
Richard Jones r1chardj0...@gmail.com writes:

 Hi all,

 At the summit last week, we developed a plan for moving forward with
 modernising Horizon's UI using AngularJS. If you weren't at that meeting
 and are interested in helping out with this effort please let me know!

I have been working on an interface for Swift written in Angular. You'll
find a ready-to-deploy tarball and zip file here:

  https://github.com/zerovm/swift-browser/releases/

This will eventually be an interface for starting ZeroVM instances on a
Swift cluster, but for now it's a pure Swift interface.

I don't know if this is something that can be used in Horizon, but maybe
there are useful bits.

One thing interesting thing in Swift Browser is its testing: I wrote a
Swift simulator that is injected into the application for end-to-end
testing with Protractor. Is is basically an in-memory (and in-browser)
Swift that responds to queries made against the Swift REST API. I've
found that very useful in the development:

  
https://github.com/zerovm/swift-browser/blob/master/app/js/test/swift-simulator.js

I don't know the other OpenStack components and their APIs very well,
but I guess that this kind of mocking might be too much for a full
OpenStack dashboard.

-- 
Martin Geisler

http://google.com/+MartinGeisler


pgpHlDlDL2w3E.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Maxime Vidori
Thanks Martin, I remember that eslint was mentioned at some point to replace 
jshint. Maybe, it can be interesting to look at this, the extension possibility 
is a major point for a switch.

- Original Message -
From: Martin Geisler mar...@geisler.net
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.org
Sent: Wednesday, November 12, 2014 10:37:15 AM
Subject: Re: [openstack-dev] [Horizon] the future of angularjs development  
in Horizon

Richard Jones r1chardj0...@gmail.com writes:

 Hi all,

 At the summit last week, we developed a plan for moving forward with
 modernising Horizon's UI using AngularJS. If you weren't at that meeting
 and are interested in helping out with this effort please let me know!

I have been working on an interface for Swift written in Angular. You'll
find a ready-to-deploy tarball and zip file here:

  https://github.com/zerovm/swift-browser/releases/

This will eventually be an interface for starting ZeroVM instances on a
Swift cluster, but for now it's a pure Swift interface.

I don't know if this is something that can be used in Horizon, but maybe
there are useful bits.

One thing interesting thing in Swift Browser is its testing: I wrote a
Swift simulator that is injected into the application for end-to-end
testing with Protractor. Is is basically an in-memory (and in-browser)
Swift that responds to queries made against the Swift REST API. I've
found that very useful in the development:

  
https://github.com/zerovm/swift-browser/blob/master/app/js/test/swift-simulator.js

I don't know the other OpenStack components and their APIs very well,
but I guess that this kind of mocking might be too much for a full
OpenStack dashboard.

-- 
Martin Geisler

http://google.com/+MartinGeisler

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Thomas Goirand
On 11/11/2014 03:02 PM, Richard Jones wrote:
 Hi all,
 
 At the summit last week, we developed a plan for moving forward with
 modernising Horizon's UI using AngularJS. If you weren't at that meeting
 and are interested in helping out with this effort please let me know!
 
 The relevant etherpad from the meeting:
 https://etherpad.openstack.org/p/kilo-horizon-contributors-meetup
 
 TL;DR: piece by piece we will replace Django views in Horizon with
 angular views, and we're going to start with Identity
 
 First up, I'd like to ask the UX folk who raised their hands in that
 meeting to indicate which of the Identity panes we should start with. I
 believe a wizard was mentioned, as a way to exercise the new wizard code
 from Maxime.
 
 At the same time, I'm looking at updating the AngularJS recommendations
 in the wiki. I believe other aspects of the current approach to angular
 code should also be revisited, if we're to scale up to the full angular
 front-end envisaged. I'd appreciate if those interested in this aspect
 in particular could contact me so we can sort this out as a team!
 
 I'd like to start the design work for the new REST API layer we'll be
 exposing to the angular application code, but that is also part of the
 broader discussion about the structure of the angular code in the
 Horizon application as mentioned above. Should it be a new blueprint/spec?
 
 There were some discussions around tooling. We're using xstatic to
 manage 3rd party components, but there's a lot missing from that
 environment. I hesitate to add supporting xstatic components on to the
 already large pile of work we have to do, so would recommend we switch
 to managing those components with bower instead. For reference the list
 of 3rd party components I used in angboard* (which is really only a
 teensy fraction of the total application we'd end up with, so this
 components list is probably reduced):
 
 json3
 es5-shim
 angular
 angular-route
 angular-cookies
 angular-animate
 angular-sanitize
 angular-smart-table
 angular-local-storage
 angular-bootstrap
 angular-translate
 font-awesome
 boot
 underscore
 ng-websocket

Hi,

Let's please continue to use xstatic packages, which are very good for
distributions. I will do the effort on the Debian side of things to make
sure that we have all packaged correctly. Using some magic that do the
same kind of horrible stuff as pip install or pear install or CPAN,
or... you name it, will not fix things on my side. Quite the opposite,
it will add issues after issues. The XStatic things, on the other hand,
is very easy for me to maintain, even though that's a lot of work.

On 11/12/2014 04:03 PM, Matthias Runge wrote:
 There are companies out there, where security policy does not allow to
 install software from a third party repository. pypi etc. are
 considered as a third party here in this context.

It's also not Debian policy compliant to download from these sources in
a package. So that'd be a no-go for Debian, and anyway, these
dependencies would have to be packaged in some way.

Again, let's please stick on the xstatic way of doing things. It worked
for Juno, let's make it work for Kilo as well.

Cheers,

Thomas Goirand (zigo)


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Monty Taylor
On 11/12/2014 02:17 AM, Matthias Runge wrote:
 On 11/11/14 10:53, Jiri Tomasek wrote:
 Hey,

 Thanks for writing this up!
 
 The Storyboard project has successfully integrated these tools into
 the OpenStack CI environment.
 
 OpenStack CI and distributors are different, because OpenStack CI does
 not distribute software.


 Using javascript tooling (yeoman, grunt, bower, etc.) has this issue of
 being dependent on nodejs which if I recall correctly is causing
 problems for packagers as some versions of these tools require different
 nodejs versions - please Mathias correct me if I am wrong. I know this
 discussion has been here before, but using these tools is necessary for
 effective development. So we need to resolve the problem asap.
 Storyboard does not have this issue as it is infra thing.
 
 As far as I know, those tools don't require different nodejs versions.
 But: we can not have different node.js versions installed at the same
 time. I assume, this is true for all distributions. Creating and
 maintaining parallel installable versions just sucks and causes many issues.
 
 In the past, customers wanted the complete tool-chain to modify their
 version of dashboard. I understand, this is not an issue, for all
 companies not distributing software, but directly providing services
 based on that software.
 
 I will have to go through all dependencies and do a review, if those are
 acceptable for inclusion e.g in Fedora. The same is true for Thomas
 Goirand for inclusion in Debian.
 

 Petr Belanyi has added optional jshint install for js linting into
 Horizon and it installs nodejs as it depends on it. Could this approach
 work for our need of js tooling too? [1]
 
 Sigh, this nonsense doesn't go away? This is the third time the same
 issue comes up.
 
 jshint is NOT free software.
 
 https://github.com/jshint/jshint/blob/master/src/jshint.js#L19

Reasonable people disagree on this point. I, for one, am one of them. In
my opnion:

A usage clause in a license header is bonghits and unenforcable, as
copyright terms cover the legality of copying things, not how you use
them. For those terms to be enforcable, it would need to be a EULA,
which jshint does not have.

However, other people disagree with me, which is their prerogative.

I'm mainly suggesting that it's probably not nonsense, it's a
disagreement, and it's also probably not going to go away, since jshint
is far and away the most common javascript linting tool in existence.
Maybe some of the people who are vehemently opposed to the clause in the
license header should figure out an alternative.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Monty Taylor
On 11/12/2014 02:40 AM, Richard Jones wrote:
 On 12 November 2014 18:17, Matthias Runge mru...@redhat.com wrote:
 
 On 11/11/14 10:53, Jiri Tomasek wrote:
 Hey,

 Thanks for writing this up!

 The Storyboard project has successfully integrated these tools into
 the OpenStack CI environment.

 OpenStack CI and distributors are different, because OpenStack CI does
 not distribute software.

 
 Ah, I wasn't clear; my concern was whether the tools chosen would be
 compatible with the CI environment. I'm hoping that distribution of the
 tools isn't our concern (see below).
u
 Using javascript tooling (yeoman, grunt, bower, etc.) has this issue of
 being dependent on nodejs which if I recall correctly is causing
 problems for packagers as some versions of these tools require different
 nodejs versions - please Mathias correct me if I am wrong. I know this
 discussion has been here before, but using these tools is necessary for
 effective development. So we need to resolve the problem asap.
 Storyboard does not have this issue as it is infra thing.

 As far as I know, those tools don't require different nodejs versions.
 But: we can not have different node.js versions installed at the same
 time. I assume, this is true for all distributions. Creating and
 maintaining parallel installable versions just sucks and causes many
 issues.

 
 I believe the nodeenv method of installing node solves this, as it's
 entirely local to the development environment.
 

Just for the record, I believe that we should chose the tools that make
sense for making our software, as long as it's not physically impossible
for them to be packaged. This means we should absolutely not use things
that require multiple versions of node to be needed. The nodejs that's
in trusty is new enough to work with all of the modern javascript tool
chain things needed for this, so other than the various javascript tools
and libraries not being packaged in the distros yet, it should be fine.

That a bunch of javascript libraries will need to be distro pacakged
should not be a blocker (although I don't think that anyone is saying it
is) That is, after all, the important work the distros do. At this
point, given the popularity of javascript and javascript tooling, I'm
pretty sure the problem is going to have to be solved at some point.

 I will have to go through all dependencies and do a review, if those are
 acceptable for inclusion e.g in Fedora. The same is true for Thomas
 Goirand for inclusion in Debian.


 Petr Belanyi has added optional jshint install for js linting into
 Horizon and it installs nodejs as it depends on it. Could this approach
 work for our need of js tooling too? [1]

 Sigh, this nonsense doesn't go away? This is the third time the same
 issue comes up.

 jshint is NOT free software.

 https://github.com/jshint/jshint/blob/master/src/jshint.js#L19
 https://github.com/jshint/jshint/blob/master/src/jshint.js#L19
 
 
 They're trying to resolve that https://github.com/jshint/jshint/issues/1234
 
 But regardless, jshint doesn't have to be installed from a Linux
 repository; it's usually installed using npm alongside the other node tools.
 
 
 Richard
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Jiri Tomasek

On 11/12/2014 02:35 PM, Monty Taylor wrote:

On 11/12/2014 02:40 AM, Richard Jones wrote:

On 12 November 2014 18:17, Matthias Runge mru...@redhat.com wrote:


On 11/11/14 10:53, Jiri Tomasek wrote:

Hey,

Thanks for writing this up!

The Storyboard project has successfully integrated these tools into
the OpenStack CI environment.

OpenStack CI and distributors are different, because OpenStack CI does
not distribute software.


Ah, I wasn't clear; my concern was whether the tools chosen would be
compatible with the CI environment. I'm hoping that distribution of the
tools isn't our concern (see below).

u

Using javascript tooling (yeoman, grunt, bower, etc.) has this issue of
being dependent on nodejs which if I recall correctly is causing
problems for packagers as some versions of these tools require different
nodejs versions - please Mathias correct me if I am wrong. I know this
discussion has been here before, but using these tools is necessary for
effective development. So we need to resolve the problem asap.
Storyboard does not have this issue as it is infra thing.

As far as I know, those tools don't require different nodejs versions.
But: we can not have different node.js versions installed at the same
time. I assume, this is true for all distributions. Creating and
maintaining parallel installable versions just sucks and causes many
issues.


I believe the nodeenv method of installing node solves this, as it's
entirely local to the development environment.


Just for the record, I believe that we should chose the tools that make
sense for making our software, as long as it's not physically impossible
for them to be packaged. This means we should absolutely not use things
that require multiple versions of node to be needed. The nodejs that's
in trusty is new enough to work with all of the modern javascript tool
chain things needed for this, so other than the various javascript tools
and libraries not being packaged in the distros yet, it should be fine.

That a bunch of javascript libraries will need to be distro pacakged
should not be a blocker (although I don't think that anyone is saying it
is) That is, after all, the important work the distros do. At this
point, given the popularity of javascript and javascript tooling, I'm
pretty sure the problem is going to have to be solved at some point.

+1, I am really glad this has been said.



I will have to go through all dependencies and do a review, if those are

acceptable for inclusion e.g in Fedora. The same is true for Thomas
Goirand for inclusion in Debian.


Petr Belanyi has added optional jshint install for js linting into
Horizon and it installs nodejs as it depends on it. Could this approach
work for our need of js tooling too? [1]

Sigh, this nonsense doesn't go away? This is the third time the same
issue comes up.

jshint is NOT free software.

https://github.com/jshint/jshint/blob/master/src/jshint.js#L19
https://github.com/jshint/jshint/blob/master/src/jshint.js#L19


They're trying to resolve that https://github.com/jshint/jshint/issues/1234

But regardless, jshint doesn't have to be installed from a Linux
repository; it's usually installed using npm alongside the other node tools.


 Richard



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Jiri

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Jiri Tomasek

On 11/11/2014 08:02 AM, Richard Jones wrote:

Hi all,

At the summit last week, we developed a plan for moving forward with 
modernising Horizon's UI using AngularJS. If you weren't at that 
meeting and are interested in helping out with this effort please let 
me know!


The relevant etherpad from the meeting:
https://etherpad.openstack.org/p/kilo-horizon-contributors-meetup

TL;DR: piece by piece we will replace Django views in Horizon with 
angular views, and we're going to start with Identity


First up, I'd like to ask the UX folk who raised their hands in that 
meeting to indicate which of the Identity panes we should start with. 
I believe a wizard was mentioned, as a way to exercise the new wizard 
code from Maxime.


At the same time, I'm looking at updating the AngularJS 
recommendations in the wiki. I believe other aspects of the current 
approach to angular code should also be revisited, if we're to scale 
up to the full angular front-end envisaged. I'd appreciate if those 
interested in this aspect in particular could contact me so we can 
sort this out as a team!


I'd like to start the design work for the new REST API layer we'll be 
exposing to the angular application code, but that is also part of the 
broader discussion about the structure of the angular code in the 
Horizon application as mentioned above. Should it be a new blueprint/spec?


There were some discussions around tooling. We're using xstatic to 
manage 3rd party components, but there's a lot missing from that 
environment. I hesitate to add supporting xstatic components on to the 
already large pile of work we have to do, so would recommend we switch 
to managing those components with bower instead. For reference the 
list of 3rd party components I used in angboard* (which is really only 
a teensy fraction of the total application we'd end up with, so this 
components list is probably reduced):


json3
es5-shim
angular
angular-route
angular-cookies
angular-animate
angular-sanitize
angular-smart-table
angular-local-storage
angular-bootstrap
angular-translate
font-awesome
boot
underscore
ng-websocket

Just looking at PyPI, it looks like only a few of those are in 
xstatic, and those are out of date.


grunt provides a lot of features for developing an angular interface. 
In particular LiveReload accelerates development significantly. 
There's a django-livereload but it uses tiny-lr under the hood, so 
we're still using a node application for LiveReload support... so it 
might make sense to just use grunt. grunt provides many other features 
as well (wiredep integration with bower, build facilities with ngMin, 
test monitoring and reload etc).


There seemed to be agreement to move to jasmine (from qunit) for 
writing the tests. It's not noted in the etherpad, but I recall karma 
was accepted as a given for the test runner. For those not in the 
meeting, angboard uses mocha+chai for test writing, but I agreed that 
jasmine is acceptable, and is already used by Storyboard (see below).


Also, phantomjs so we don't have to fire up a browser for exercising 
(what should hopefully be an extensive) unit test suite.


The Storyboard project has successfully integrated these tools into 
the OpenStack CI environment.



 Richard

* https://github.com/r1chardj0n3s/angboard


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



I am going to try to conclude what has been said in emails across this 
thread.


As Monty Taylor said, nodejs itself is not a blocker as multiple 
versions of it should not be needed by our tools. (That's also what npm 
and bower are taking care of, right?) Only thing that is required is 
that all tools/js libs we want to use would eventually have to be 
packaged. This is just a bunch of work for packagers.



Approach on using Xstatic packages vs Js tooling:

As only problem with using js tooling should be just actual packaging of 
it, I think it makes sense to use these tools and make development 
simpler then going other way around and using Xstatic packages - which 
means devs would have to care about getting stuff packaged as xstatic 
and added to the code, while maintaining proper versions and making sure 
that they work ok together. NPM and Bower do this for us. Common sense 
tells me packagers should take care of packaging.
Packaging of these tools will have to get resolved somehow anyway, as 
there will be rise in requirements of using them not just from Horizon...



Which tools should we use eventually:

Based on the contributions by Maxime, Martin and the others, I think the 
list of tools should end up as follows:


Tooling:
npm
bower
gulp
Jasmine
Karma/Protractor(?)/eslint
...?

Bower and Gulp seems to get along well 
(https://github.com/yeoman/generator-gulp-webapp)


Tastypie on the Django side

Angular 

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Julie Pichon
On 12/11/14 15:12, Jiri Tomasek wrote:
 Approach on using Xstatic packages vs Js tooling:
 
 As only problem with using js tooling should be just actual packaging of
 it, I think it makes sense to use these tools and make development
 simpler then going other way around and using Xstatic packages - which
 means devs would have to care about getting stuff packaged as xstatic
 and added to the code, while maintaining proper versions and making sure
 that they work ok together. NPM and Bower do this for us. Common sense
 tells me packagers should take care of packaging.
 Packaging of these tools will have to get resolved somehow anyway, as
 there will be rise in requirements of using them not just from Horizon...

I can't speak for the rest but that part doesn't seem correct to me. The
XStatic packages are Python packages (as in, dependencies) that the
Horizon team is responsible for (when they don't already exist) and
maintains on stackforge, so we do have to create them and make sure they
all work well together. The later packaging as rpm or deb or others is
left to the distro packagers of course.

There are instructions already on how to create xstatic packages [1],
it's not very complicated and just takes some review time.

Thanks,

Julie

[1]
http://docs.openstack.org/developer/horizon/contributing.html#javascript-and-css-libraries


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Jiri Tomasek

On 11/12/2014 05:18 PM, Julie Pichon wrote:

On 12/11/14 15:12, Jiri Tomasek wrote:

Approach on using Xstatic packages vs Js tooling:

As only problem with using js tooling should be just actual packaging of
it, I think it makes sense to use these tools and make development
simpler then going other way around and using Xstatic packages - which
means devs would have to care about getting stuff packaged as xstatic
and added to the code, while maintaining proper versions and making sure
that they work ok together. NPM and Bower do this for us. Common sense
tells me packagers should take care of packaging.
Packaging of these tools will have to get resolved somehow anyway, as
there will be rise in requirements of using them not just from Horizon...

I can't speak for the rest but that part doesn't seem correct to me. The
XStatic packages are Python packages (as in, dependencies) that the
Horizon team is responsible for (when they don't already exist) and
maintains on stackforge, so we do have to create them and make sure they
all work well together. The later packaging as rpm or deb or others is
left to the distro packagers of course.

There are instructions already on how to create xstatic packages [1],
it's not very complicated and just takes some review time.

Thanks,

Julie

[1]
http://docs.openstack.org/developer/horizon/contributing.html#javascript-and-css-libraries


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


I might have expressed myself wrong about XStatic packages. But as you 
say as well, to use XStatic packages, we need to most often create them 
and maintain the correct versions we require in Horizon and they don't 
help to packagers either.  I makes sense to use them in Django 
application as they can be included in requirements.txt and we don't 
have to carry them directly in the code. So I am definitely ok to use 
them for Django dependencies we have.


Similar thing is npm and bower doing on the Angular side except for we 
don't have to create these libraries as they already exist. NPM and 
Bower are taking care of including the right versions of js libs our dev 
env and our application needs. They use similar description files as 
requirements.txt in Django.


It makes no sense not to use them and try to include js libraries using 
XStatic packages and listing them in requirements.txt because this way 
we don't know which version of js lib to use etc. NPM and bower are 
doing this for us.


In both approaches dependencies need to have packages in the end 
regadles of being it XStatic package or js library or Angular module.


It is about using the right tools for the job.


I see relation between Nodejs and js libs/tools and Angular app defining 
it's dependencies using NPM and Bower quite similar as Ruby, Rubygems 
and Rails application defining it's dependencies in Gemfile.lock. 
Rubygems are being packaged in distros, so why shouldn't node packages?



Jiri


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Adam Young

On 11/12/2014 03:03 AM, Matthias Runge wrote:

On 12/11/14 08:40, Richard Jones wrote:


I believe the nodeenv method of installing node solves this, as it's
entirely local to the development environment.

See below, this touches package build as well.


 I will have to go through all dependencies and do a review, if those are
 acceptable for inclusion e.g in Fedora. The same is true for Thomas
 Goirand for inclusion in Debian.

 
  Petr Belanyi has added optional jshint install for js linting into
  Horizon and it installs nodejs as it depends on it. Could this approach
  work for our need of js tooling too? [1]

 Sigh, this nonsense doesn't go away? This is the third time the same
 issue comes up.

 jshint is NOT free software.

 https://github.com/jshint/jshint/blob/master/src/jshint.js#L19


They're trying to resolve that https://github.com/jshint/jshint/issues/1234

But regardless, jshint doesn't have to be installed from a Linux
repository; it's usually installed using npm alongside the other node tools.


Thanks for the pointer, this is good news!

Regarding package managers, my POV is completely different. From a
distributor perspective, where customers expect everything provided from
a single source, I'm not using npm, pip etc. I'm packaging that stuff
properly before using it.

There are companies out there, where security policy does not allow to
install software from a third party repository. pypi etc. are considered
as a third party here in this context.

I would prefer to have the complete tool chain available as (RPM)
packages. I am executing unit tests etc. during package build. Our
builders don't have access to the internet, downloading any other stuff
from the internet is no option.


And this is really not-negotiable, too.  Debian has the same 
requirements, it is not Red Hat/Fedora speciufic.  It is no different 
than the Python Code.  We dodn't pip install for RHOSP for the Python 
packages, and we can't use any of the language specific package managers.


But, we are used to it, and dealing with package dependencies is what we 
do.  Having these in Fedora, while painful, will be ultimately very 
valuable.




Matthias





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Richard Jones
On 13 November 2014 09:35, Adam Young ayo...@redhat.com wrote:

 On 11/12/2014 03:03 AM, Matthias Runge wrote:

 On 12/11/14 08:40, Richard Jones wrote:

  I believe the nodeenv method of installing node solves this, as it's
 entirely local to the development environment.

 See below, this touches package build as well.


  I will have to go through all dependencies and do a review, if
 those are
  acceptable for inclusion e.g in Fedora. The same is true for Thomas
  Goirand for inclusion in Debian.

  
   Petr Belanyi has added optional jshint install for js linting into
   Horizon and it installs nodejs as it depends on it. Could this
 approach
   work for our need of js tooling too? [1]

  Sigh, this nonsense doesn't go away? This is the third time the same
  issue comes up.

  jshint is NOT free software.

  https://github.com/jshint/jshint/blob/master/src/jshint.js#L19


 They're trying to resolve that https://github.com/jshint/
 jshint/issues/1234

 But regardless, jshint doesn't have to be installed from a Linux
 repository; it's usually installed using npm alongside the other node
 tools.

  Thanks for the pointer, this is good news!

 Regarding package managers, my POV is completely different. From a
 distributor perspective, where customers expect everything provided from
 a single source, I'm not using npm, pip etc. I'm packaging that stuff
 properly before using it.

 There are companies out there, where security policy does not allow to
 install software from a third party repository. pypi etc. are considered
 as a third party here in this context.

 I would prefer to have the complete tool chain available as (RPM)
 packages. I am executing unit tests etc. during package build. Our
 builders don't have access to the internet, downloading any other stuff
 from the internet is no option.


 And this is really not-negotiable, too.  Debian has the same requirements,
 it is not Red Hat/Fedora speciufic.  It is no different than the Python
 Code.  We dodn't pip install for RHOSP for the Python packages, and we
 can't use any of the language specific package managers.

 But, we are used to it, and dealing with package dependencies is what we
 do.  Having these in Fedora, while painful, will be ultimately very
 valuable.


I guess a point of difference here is that the Linux packaging systems
target global installation, whereas bower and npm (in our case) target a
local installation. It's effectively vendoring the code, but you still have
to pull it down from a repository for each installation. Don't worry, I'm
not going to suggest we actually move back to vendoring :)

We're currently using xstatic and that works with Linux packaging because
it was designed to cope with being a global installation. The current
Horizon codebase has a django-xstatic plugin which further makes dealing
with xstatic components nicer - for example it handles path management and
static file compilation (JS minification and concatenation, for example).
That's really nice, but poses some problems:

- we would need to xstatic-ify (and deb/rpm-ify) all those components
- we could run into global version conflict issues if we run more than one
service on a system - is this likely to be an issue in practise though?
- as far as I'm aware, the xstatic JS minification is not angular-aware,
and will break angular code that has not been written to be
dumb-minifier-aware (the angular minifier ngMin is written in node and
knows how to do things more correctly); adding dumb-minifier-awareness to
angular code makes it ugly and more error-prone :/

On the minification issue though: I talked to a few people at the summit
and we agreed that minification should just be avoided; gzip at the
transport layer buys significantly better compression, and has the added
bonus that the resultant code is readable. I note that the Debian JS
guidelines* only recommend that libraries *should* be minified (though I'm
unsure why they even recommend that).

One further note on xstatic vs. bower: during development it's very useful
to be able to install random components from bower to try them out; during
angboard development I would sometimes install and remove half a dozen
components just in one day. I would hope that whatever solution we come up
with has at least some of this flexibility for experimentation.


Richard

* https://wiki.debian.org/Javascript/Policy
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Thomas Goirand
On 11/12/2014 11:12 PM, Jiri Tomasek wrote:
 As Monty Taylor said, nodejs itself is not a blocker as multiple
 versions of it should not be needed by our tools. (That's also what npm
 and bower are taking care of, right?) Only thing that is required is
 that all tools/js libs we want to use would eventually have to be
 packaged. This is just a bunch of work for packagers.
 
 Approach on using Xstatic packages vs Js tooling:
 
 As only problem with using js tooling should be just actual packaging of
 it, I think it makes sense to use these tools and make development
 simpler then going other way around and using Xstatic packages - which
 means devs would have to care about getting stuff packaged as xstatic
 and added to the code, while maintaining proper versions and making sure
 that they work ok together. NPM and Bower do this for us. Common sense
 tells me packagers should take care of packaging.
 Packaging of these tools will have to get resolved somehow anyway, as
 there will be rise in requirements of using them not just from Horizon...

I see an obvious issue with your approach.

Just like with pip and PyPi, it's going to be just one line of patch for
Horizon people to add yet-another-javascript-dependency. It's already
a dependency hell. I have created 21 new python-xstatic packages in
Debian, and at least, for half a dozen of them (maybe more), I also
packaged the libjs-* packages. It took a long time to have them in
order, and it just got stabilized for Juno (there's still something to
fix in Jessie, but I don't really care as it's Icehouse there...).

Now, your motivation to go into the direction of using tooling seems to
be motivated so that it's super easy to add more. And more... And more
again. This leads me to believe that it's going to be even more crazy.
When is this going to stop? We're definitively enlarging the potential
surface of attack and potential security breaches. And OpenStack at
large is not at all doing great in terms of security. [1]

I just don't understand why all of this is needed. I did some JS
programming myself back in the days (10 years ago, using PHP...). I
could do Ajax by hand, without using a single library. What is it that
you're doing in Horizon that needs so many libs? When I see Horizon in
Juno, I don't even get it. Or is it because you just want to have fancy
animation? Frankly, I don't care these. I care a lot more that we're not
adding new potential security problems.

On 11/13/2014 12:18 AM, Julie Pichon wrote:
 There are instructions already on how to create xstatic packages [1],
 it's not very complicated and just takes some review time.

Exactly!!!

And if someone can't make the effort to package something as an XStatic
lib, then he doesn't deserve the rights to add a new libjs depends on
the project. :)

Also, why do we need to:
1/ Change our procedure starting on each new cycle
2/ Constantly reinvent the wheel
3/ Make my life miserable... :)

On 11/13/2014 01:23 AM, Jiri Tomasek wrote:
 I might have expressed myself wrong about XStatic packages. But as you
 say as well, to use XStatic packages, we need to most often create
 them and maintain the correct versions we require in Horizon and they
 don't help to packagers either.

They do! Well, except Ubuntu guys who found it funny to re-embed all the
Javascript back in Horizon, but for all sane people doing packaging in
this world, it's very helpful!!!

 Similar thing is npm and bower doing on the Angular side except for we
 don't have to create these libraries as they already exist. NPM and
 Bower are taking care of including the right versions of js libs our
 dev env and our application needs. They use similar description files
 as requirements.txt in Django.

How are you going to express the changes in the global-requirements.txt?
Is this just going to live within Horizon? How about projects like
tuskar-ui, or murano-dashboard? There simply wont be a single unique
place for me to look at to see what work I need to process. And no
unique place either to have an overview and see how bad we're doing in
terms of dependency (ie: we have a way too many...).

 It makes no sense not to use them and try to include js libraries
 using XStatic packages and listing them in requirements.txt because
 this way we don't know which version of js lib to use etc.

Not correct. XStatic package versions are following the one of upstream
libjs packages. For example, python-xstatic-jsencrypt in Debian has
version 2.0.0.2, and depends on version 2.0.0 of jsencrypt. If there's
evolution of the dependency, then the XStatic package version will also
change. I think it's very easy to follow, and a very good idea.

The thing is, using all these NodeJS stuff, we'll end-up depending on
Node so much, that it's going to pull about a 100 more dependencies (or
even more). Maybe it's going to be only build-depends, but that's the
same from my point of view: it's going to be packages that we'll need to
take care of in the distributions, 

Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Donald Stufft

 On Nov 12, 2014, at 8:29 PM, Thomas Goirand z...@debian.org wrote:
 
 On 11/12/2014 09:31 PM, Monty Taylor wrote:
 jshint is NOT free software.
 
 https://github.com/jshint/jshint/blob/master/src/jshint.js#L19
 Reasonable people disagree on this point.
 
 Feel free to have this debate with the entire Debian community. When
 you're done, then come back to us, and we can use jshint. In the mean
 while, let's not use it. (and by the way, read again the Debian Free
 Software Guideline and especially point number 5 and 6, I'm not saying
 that this is *my* point of argumentation, but it has been so within some
 threads in Debian).

The terrible line in the jshint license is going to go away in the future,
you can read https://github.com/jshint/jshint/issues/1234#issuecomment-56875247
but the tl;dr is that Doug Crockford gave Eclipse a license to his software
without that line, and eclipse is sharing it so jshint is going to rebase off
of the “Crockford” version and onto the Eclipse version.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon

2014-11-12 Thread Gabriel Hurley
Two things of note, having been doing heavy javascript development over the 
past couple years:

1) NPM actually resolves conflicts in a dependency tree. Unlike Python, it can 
ensure that if different packages declare conflicting versions, each one gets 
the version it requested. And conflicting dependencies are very common in JS 
packages. That's gonna be a nightmare for more traditional packaging systems 
if you try to shoehorn them together.

2) The OpenStack community is overwhelmingly fond of reinventing wheels. The 
JavaScript community has the opposite habit, and tends to create a package for 
every last function (that's why some of these seemingly single-purpose tools 
have 50+ dependencies). I caution against putting in place a system that 
penalizes developers for trying to use established tools or to use packages 
that do the job and do it well (such as forcing them to deal with xstatic 
packaging). While I respect the licensing and packaging concerns involved in 
large dependency trees, we need to stop encouraging people to reinvent wheels.

I think most folks at this point agree that the future of Horizon is to become 
a JavaScript-driven web app. It's the way the industry is going this point, and 
it's the way to satisfy what users want and expect. Provided we accept that 
future, we should strive to embrace the best practices of that development 
community, not to tell them we don't like it or it doesn't fit how OpenStack 
does things.

On a historical note, OpenStack had a very rocky relationship with the broader 
Python community in its early days. Since then, thanks to great efforts on both 
sides, that relationship has gotten much better. Let's try not to replay 
history by doing the same thing with the JavaScript/Node community.

We have to attract great developers so we can produce the best possible 
OpenStack. I remind people to think about how we can do *that* first and how we 
can deal with distro requirements to support the process second.

- Gabriel


-Original Message-
From: Thomas Goirand [mailto:z...@debian.org] 
Sent: Wednesday, November 12, 2014 5:30 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Horizon] the future of angularjs development in 
Horizon

On 11/12/2014 11:12 PM, Jiri Tomasek wrote:
 As Monty Taylor said, nodejs itself is not a blocker as multiple 
 versions of it should not be needed by our tools. (That's also what 
 npm and bower are taking care of, right?) Only thing that is required 
 is that all tools/js libs we want to use would eventually have to be 
 packaged. This is just a bunch of work for packagers.
 
 Approach on using Xstatic packages vs Js tooling:
 
 As only problem with using js tooling should be just actual packaging 
 of it, I think it makes sense to use these tools and make development 
 simpler then going other way around and using Xstatic packages - which 
 means devs would have to care about getting stuff packaged as xstatic 
 and added to the code, while maintaining proper versions and making 
 sure that they work ok together. NPM and Bower do this for us. Common 
 sense tells me packagers should take care of packaging.
 Packaging of these tools will have to get resolved somehow anyway, as 
 there will be rise in requirements of using them not just from Horizon...

I see an obvious issue with your approach.

Just like with pip and PyPi, it's going to be just one line of patch for 
Horizon people to add yet-another-javascript-dependency. It's already a 
dependency hell. I have created 21 new python-xstatic packages in Debian, and 
at least, for half a dozen of them (maybe more), I also packaged the libjs-* 
packages. It took a long time to have them in order, and it just got stabilized 
for Juno (there's still something to fix in Jessie, but I don't really care as 
it's Icehouse there...).

Now, your motivation to go into the direction of using tooling seems to be 
motivated so that it's super easy to add more. And more... And more again. This 
leads me to believe that it's going to be even more crazy.
When is this going to stop? We're definitively enlarging the potential surface 
of attack and potential security breaches. And OpenStack at large is not at all 
doing great in terms of security. [1]

I just don't understand why all of this is needed. I did some JS programming 
myself back in the days (10 years ago, using PHP...). I could do Ajax by hand, 
without using a single library. What is it that you're doing in Horizon that 
needs so many libs? When I see Horizon in Juno, I don't even get it. Or is it 
because you just want to have fancy animation? Frankly, I don't care these. I 
care a lot more that we're not adding new potential security problems.

On 11/13/2014 12:18 AM, Julie Pichon wrote:
 There are instructions already on how to create xstatic packages [1], 
 it's not very complicated and just takes some review time.

Exactly!!!

And if someone can't make

  1   2   >