That was also required for bro 2.5.2, so I did that here <https://github.com/apache/metron/commit/59fe1b453279bf5c7df627ea656c762b3a98e777>. Feel free to reuse the approach elsewhere
Jon On Mon, Nov 27, 2017 at 10:03 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > First issue is that we need c++ 11 on centos 6.8 > > > > On November 27, 2017 at 09:53:55, Simon Elliston Ball ( > si...@simonellistonball.com) wrote: > > Well, that’s good news on that issue. Reproducing the problem is half way > to solving it, right? > > I would still say there are some systemic things going on that have > manifested in a variety of ways on both the users and dev list, so it’s > worth us having a good look at a more robust approach to node dependencies > (both npm ones, and the native ones) > > Simon > > On 27 Nov 2017, at 13:30, Otto Fowler <ottobackwa...@gmail.com> wrote: > > I can reproduce the failure in out ansible docker build container, which is > also centos. > The issue is building our node on centos in all these cases. > > > > On November 27, 2017 at 07:02:51, Simon Elliston Ball ( > si...@simonellistonball.com) wrote: > > Thinking about this, doesn’t our build plugin explicitly install it’s own > node? So actually all the node version things may be a red herring, since > this is under our control through the pom. Not sure if we actually > exercising this control. It seems that some of the errors people report are > more to do with compilation failures for native node modules, which is > doesn’t pin (i.e. things like system library dependencies). I’m not sure > what we have in the dependency tree that requires complex native > dependencies, but this might just be one of those node things we could doc > around. > > This scenario would be fixed by standardising the build container. > > Yarn’s big thing is that it enables faster dependency resolution and local > caching, right? It does not seem to address any of the problems we see, but > sure, it’s the shiny new dependency system for node modules, which might > make npm less horrible to deal with, so worth looking into. > > The other issue that I’ve seen people run into a lot is flat out download > errors. This could be helped by finding our versions, maybe with yarn, but > let’s face it, package-lock.json could also do that with npm, albeit with a > slightly slower algorithm. However, short of bundling and hosting deps > ourselves, I suspect the download errors are beyond our control, and > certainly beyond the scope of this project (fix maven, fix npm, fix all the > node hosting servers…) > > Simon > > > > On 27 Nov 2017, at 07:28, RaghuMitra Kandikonda < > raghumitra....@gmail.com> > wrote: > > > > Looking at some of the build failure emails and past experience i > > would suggest having a node & npm version check in our build scripts > > and moving dependency management to yarn. > > > > We need not restrict the build to a specific version of node & npm but > > we can surely suggest a min version required to build UI successfully. > > > > -Raghu > > > > > > > > On Fri, Nov 24, 2017 at 10:21 PM, Simon Elliston Ball > > <si...@simonellistonball.com> wrote: > >> Agreeing with Nick, it seems like the main reason people are building > themselves, and hitting all these environmental issues, is that we do not > as a project produce binary release artefacts (the rpms which users could > just install) and instead leave that for the commercial distributors to do. > >> > >> Yarn may help with some of the dependency version issues we’re having, > but not afaik with the core missing library headers / build tools / node > and npm version issue, those would seem to fit a documentation fix and > improvements to platform-info to flag the problems, so this can then be a > pre-flight check tool as well as a diagnostic tool. > >> > >> Another option I would put on the table is to standardise our build > environment, so that the non-java bits are run in a standard docker image > or something fo the sort, that way we can take control of all the > environmental and OS dependent pieces, much as we do right now with the rpm > build sections of the mpack build. > >> > >> The challenge here will be adding the relevant maven support. At the > moment we’re relying on the maven npm and node build plugins, this would > likely need replacing with something custom and a challenge to support to > go dow this route. > >> > >> Perhaps the real answer here is to push people who are just kicking the > tyres towards a binary distribution, or at least rpm artefacts as part of > the Apache release to give them a head start for a happy path on a known > good OS environment. > >> > >> Simon > >> > >>> On 24 Nov 2017, at 16:01, Nick Allen <n...@nickallen.org> wrote: > >>> > >>> Yes, it is a problem. I think you've identified a couple important > things > >>> that we could address in parallel. I see these as challenges we need to > >>> solve for the dev community. > >>> > >>> (1) NPM is causing us some major headaches. Which version do we > require? > > >>> How do I install that version (on Mac, Windows, Linux)? Does YARN help > >>> here at all? > >>> > >>> (2) Can we automate the prerequisite checks that we currently do > manually > >>> with `platform-info.sh`? An automated check could run and fail as part > of > >>> the build or deployment process. > >>> > >>> > >>> > >>> More importantly though is that users should not have to build Metron > at > > >>> all. They should not have to worry about installing NPM and the rest of > >>> the development tooling. Here are some options that are not mutually > >>> exclusive. > >>> > >>> > >>> (1) Create an image in Atlas that has Metron fully installed. A new > user > > >>> could run single node Metron on their laptop with a single command and > the > >>> only prereqs would be Vagrant and Virtualbox. We could cut new images > for > >>> each Metron release. Or selectively cut new dev images from master as > we > > >>> see fit. > >>> > >>> (2) Distribute the Metron RPMs (and the MPack tarball?) so that users > can > >>> install Metron on a cluster without having to build it. > >>> > >>> > >>> > >>> > >>> > >>> > >>> On Fri, Nov 24, 2017 at 10:11 AM, Otto Fowler <ottobackwa...@gmail.com > > > >>> wrote: > >>> > >>>> It seems like it is getting *very* common for people to have trouble > >>>> building recently. Errors with NPM and Node seen common, with fixes > ranging > >>>> from updating c/c++ libs to the version of npm/node. > >>>> > >>>> There has to be a better way to do this. > >>>> > >>>> - > >>>> > >>>> Are we out of date or missing requirements in our documentation? > >>>> - > >>>> > >>>> Does our documentation need to be updated for building? > >>>> - > >>>> > >>>> Is there a better way in maven to check the versions required for some > >>>> of these things and fail faster with a better message? > >>>> - > >>>> > >>>> Are we building correctly or are we asking for trouble? > >>>> > >>>> The ability to build metron is pretty important, and it seems that > people > >>>> are having a lot of trouble related to the new technologies in alerts > and > >>>> config ui. > >>>> > >> > -- Jon
