Well, that’s good news on that issue. Reproducing the problem is half way to solving it, right?
I would still say there are some systemic things going on that have manifested in a variety of ways on both the users and dev list, so it’s worth us having a good look at a more robust approach to node dependencies (both npm ones, and the native ones) Simon > On 27 Nov 2017, at 13:30, Otto Fowler <ottobackwa...@gmail.com> wrote: > > I can reproduce the failure in out ansible docker build container, which is > also centos. > The issue is building our node on centos in all these cases. > > > > On November 27, 2017 at 07:02:51, Simon Elliston Ball > (si...@simonellistonball.com <mailto:si...@simonellistonball.com>) wrote: > >> Thinking about this, doesn’t our build plugin explicitly install it’s own >> node? So actually all the node version things may be a red herring, since >> this is under our control through the pom. Not sure if we actually >> exercising this control. It seems that some of the errors people report are >> more to do with compilation failures for native node modules, which is >> doesn’t pin (i.e. things like system library dependencies). I’m not sure >> what we have in the dependency tree that requires complex native >> dependencies, but this might just be one of those node things we could doc >> around. >> >> This scenario would be fixed by standardising the build container. >> >> Yarn’s big thing is that it enables faster dependency resolution and local >> caching, right? It does not seem to address any of the problems we see, but >> sure, it’s the shiny new dependency system for node modules, which might >> make npm less horrible to deal with, so worth looking into. >> >> The other issue that I’ve seen people run into a lot is flat out download >> errors. This could be helped by finding our versions, maybe with yarn, but >> let’s face it, package-lock.json could also do that with npm, albeit with a >> slightly slower algorithm. However, short of bundling and hosting deps >> ourselves, I suspect the download errors are beyond our control, and >> certainly beyond the scope of this project (fix maven, fix npm, fix all the >> node hosting servers…) >> >> Simon >> >> >> > On 27 Nov 2017, at 07:28, RaghuMitra Kandikonda <raghumitra....@gmail.com >> > <mailto:raghumitra....@gmail.com>> wrote: >> > >> > Looking at some of the build failure emails and past experience i >> > would suggest having a node & npm version check in our build scripts >> > and moving dependency management to yarn. >> > >> > We need not restrict the build to a specific version of node & npm but >> > we can surely suggest a min version required to build UI successfully. >> > >> > -Raghu >> > >> > >> > >> > On Fri, Nov 24, 2017 at 10:21 PM, Simon Elliston Ball >> > <si...@simonellistonball.com <mailto:si...@simonellistonball.com>> wrote: >> >> Agreeing with Nick, it seems like the main reason people are building >> >> themselves, and hitting all these environmental issues, is that we do not >> >> as a project produce binary release artefacts (the rpms which users could >> >> just install) and instead leave that for the commercial distributors to >> >> do. >> >> >> >> Yarn may help with some of the dependency version issues we’re having, >> >> but not afaik with the core missing library headers / build tools / node >> >> and npm version issue, those would seem to fit a documentation fix and >> >> improvements to platform-info to flag the problems, so this can then be a >> >> pre-flight check tool as well as a diagnostic tool. >> >> >> >> Another option I would put on the table is to standardise our build >> >> environment, so that the non-java bits are run in a standard docker image >> >> or something fo the sort, that way we can take control of all the >> >> environmental and OS dependent pieces, much as we do right now with the >> >> rpm build sections of the mpack build. >> >> >> >> The challenge here will be adding the relevant maven support. At the >> >> moment we’re relying on the maven npm and node build plugins, this would >> >> likely need replacing with something custom and a challenge to support to >> >> go dow this route. >> >> >> >> Perhaps the real answer here is to push people who are just kicking the >> >> tyres towards a binary distribution, or at least rpm artefacts as part of >> >> the Apache release to give them a head start for a happy path on a known >> >> good OS environment. >> >> >> >> Simon >> >> >> >>> On 24 Nov 2017, at 16:01, Nick Allen <n...@nickallen.org >> >>> <mailto:n...@nickallen.org>> wrote: >> >>> >> >>> Yes, it is a problem. I think you've identified a couple important >> >>> things >> >>> that we could address in parallel. I see these as challenges we need to >> >>> solve for the dev community. >> >>> >> >>> (1) NPM is causing us some major headaches. Which version do we require? >> >>> How do I install that version (on Mac, Windows, Linux)? Does YARN help >> >>> here at all? >> >>> >> >>> (2) Can we automate the prerequisite checks that we currently do >> >>> manually >> >>> with `platform-info.sh`? An automated check could run and fail as part >> >>> of >> >>> the build or deployment process. >> >>> >> >>> >> >>> >> >>> More importantly though is that users should not have to build Metron at >> >>> all. They should not have to worry about installing NPM and the rest of >> >>> the development tooling. Here are some options that are not mutually >> >>> exclusive. >> >>> >> >>> >> >>> (1) Create an image in Atlas that has Metron fully installed. A new user >> >>> could run single node Metron on their laptop with a single command and >> >>> the >> >>> only prereqs would be Vagrant and Virtualbox. We could cut new images >> >>> for >> >>> each Metron release. Or selectively cut new dev images from master as we >> >>> see fit. >> >>> >> >>> (2) Distribute the Metron RPMs (and the MPack tarball?) so that users >> >>> can >> >>> install Metron on a cluster without having to build it. >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> On Fri, Nov 24, 2017 at 10:11 AM, Otto Fowler <ottobackwa...@gmail.com >> >>> <mailto:ottobackwa...@gmail.com>> >> >>> wrote: >> >>> >> >>>> It seems like it is getting *very* common for people to have trouble >> >>>> building recently. Errors with NPM and Node seen common, with fixes >> >>>> ranging >> >>>> from updating c/c++ libs to the version of npm/node. >> >>>> >> >>>> There has to be a better way to do this. >> >>>> >> >>>> - >> >>>> >> >>>> Are we out of date or missing requirements in our documentation? >> >>>> - >> >>>> >> >>>> Does our documentation need to be updated for building? >> >>>> - >> >>>> >> >>>> Is there a better way in maven to check the versions required for some >> >>>> of these things and fail faster with a better message? >> >>>> - >> >>>> >> >>>> Are we building correctly or are we asking for trouble? >> >>>> >> >>>> The ability to build metron is pretty important, and it seems that >> >>>> people >> >>>> are having a lot of trouble related to the new technologies in alerts >> >>>> and >> >>>> config ui. >> >>>> >> >>
