On Thu, Jul 21, 2016 at 12:46 PM, Daniel Lobato Garcia <[email protected]> wrote:
> Hi devs, > > Just wanted to give you an update on packaging for npm & webpack > incidentally. We have two PRs in core that are dabbling in a more modern > way of doing frontend development in Foreman. > https://github.com/theforeman/foreman/pull/3433 > https://github.com/theforeman/foreman/pull/3603 > > At the moment, most frontend modules are no longer served as gemified > assets but rather as npm modules. It's difficult for us to develop on a > modern stack of tools & get a nice UX experience so the idea is to move > towards a model where npm will provide all the frontend stuff you need. > Read more on: https://github.com/theforeman/rfcs/pull/3 > > -------------------------- > > Now, regarding packaging (which really has been blocking the two > aforementioned PRs and potential frontend development): > > The issue we face is that we need to serve the node_modules (folder > generated after running npm install) dependencies with Foreman somehow. > I've been trying several things, with different degrees of success. > > 1. Use npm2rpm [1] to create a package for every npm module. Only > 'direct' dependencies are specified in these packages, and they store > merely the dependency itself. > > - Upsides: > - 'the right way' according to rpm guidelines > - no confusion about dependencies and simpler RPM specs > - Downsides: > - this creates close to 1000 new packages > - we'd have to serve several different versions of the same package, > which the OS would refuse to install (e.g if we have uglify-js > 0.0.1, and > 1.0.4 in our repos, the former would be superseded by the latter) > - impossible to review this amount of packages, it must be automated > > - Success with POC? No > > 2. Create 'nodejs-bundle-xxxxx' packages which contain all of the > dependencies for a certain package. e.g webpack Provides > bundle-webpack-core, etc... > > - Upsides: > - similar to how foreman node_modules is structured in development > - Downsides: > - conflicting versions for subdependencies force us to keep track > of the 'dependency tree' in the package. For instance in this spec > > https://github.com/dLobatog/foreman-packaging/blob/5fc7bb8e831fa74f579a41e9bf9f367f6a9527ff/nodejs-webpack/nodejs-webpack.spec > we can't just unpack all 'async' to > %{buildroot}/%{nodejs_sitelib} as there are several 'async' > packages, and the last one to be unpacked will prevail. > - these packages become huge and unwieldy to review > - we have to jump hoops to make it work. for instance we have to > call '%nodejs_fixdep -r dependency' to allow dependencies to not > conflict (this doesn't always work) > - lots of 'cruft' is packed, like examples directories, etc.. > > - Success with POC? No. I stopped when I figured I had to keep track > of the dependency tree before unpackaging. > > 3. Put node_modules in a new package (foreman-npm-assets?) that > foreman-assets %install section puts in the right place? > > - Upsides: > - relatively easy to do > - jenkins can automate it > > - Downsides: > - less control or information for users about what's installed with > foreman, although we could specify the dependencies under > "Provides: bundled-npm(xxxx)" > > - Success with POC: haven't tried > > 4. Make Foreman tar file include node_modules so it's part of the > foreman package itself > - Upsides: > - easiest to do > - zero work required by devs > - closest to development > > - Downsides: > - lack of knowledge/control about deps for users, like 3. > - no possibility of depending on system nodejs-xxx packages, but I > don't think we want that. > > - Success with POC: Yes! > https://github.com/dlobatog/foreman-packaging/tree/chainbuild-webpack > is able to run webpack compile just fine after that. We'd need even > less than what's on that branch to do it. > > ---------------------------------------- > > I got kind of stuck with 2. because of the tree dependencies thing, and > honestly I don't see much of a benefit to it now, especially considering > the amount of work there's involved. > > If we can redo this somehow I'd definitely go with 4 which is really > easy and I can't see how that's a problem for us. It'd be a problem if > we were to put Foreman in a Linux distribution but we're not aiming for > that. It is as well more manageable than any other model which requires > hundreds of specs + sources in our foreman-packaging repo. > Aside from that... > > Why don't we go with 4 at least until we find a better/feasible model? It > would allow development to continue (the two PRs I mentioned + more). > There's plenty of time until 1.13 so I don't see why we can't move > forward, merge these PRs and figure out other packaging strategies if we > need to. > I would be extremely happy to move forward one way or another. I honestly want our developers to get unblocked asap. I would ask that we define npm packaging as a blocker for 1.13 release, and merge the https://github.com/theforeman/foreman/pull/3433 (or a variant of it) and would prefer option 3 than option 4 if its easily doable, as i do think foreman tar should not include 3rd party code in general. > --------------------------------------- > > If you got this far, you surely know if I'm missing something or you can > make some suggestions about which would be the best process, so please > reply. > thanks Daniel, its obvious you put a lot of effort into this, I tend to agree with your logic above, mostly because assets (javascript, css etc) gets minified/compress meaning all of your packages are build time only (they never get to users), further, since option 2 (the one we initially agreed upon in the pr discussion) already bundle some of the dependencies, I would prefer option 3 (bundle it all in one package) as does it really matters if break in into one bundle vs multiple bundles? thanks, Ohad > New ideas, comments, critiques, welcome :) > > [1] - https://github.com/dlobatog/npm2rpm > > Best, > > -- > Daniel Lobato Garcia > > @dLobatog > blog.daniellobato.me > daniellobato.me > > GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 > Keybase: https://keybase.io/elobato > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
