On Wed, Oct 6, 2021 at 2:48 AM Alexander Kanavin <[email protected]> wrote:
> The OE landscape is littered with unmaintained or barely maintained > layers. By the looks of things, no one wants to maintain this proposed > meta-nodejs layer either, which involves thousands of recipes, (so far) > non-existent tooling to create and update them in scalable ways, and > rigorous testing. Or such a layer would have already happened. > > It's for this reason that I think the shrinkwrapped, self-contained recipe > approach is more viable (even if it comes with its own pain points) and > also matches how node.js community operates - we can fix the CVE stuff as > well to look into dependencies if that's a concern. > > Alex > > On Wed, 6 Oct 2021 at 11:43, Konrad Weihmann <[email protected]> > wrote: > >> >> >> On 06.10.21 11:18, Stefan Herbrechtsmeier wrote: >> > Hi Konrad, >> > >> > Am 05.10.2021 um 21:29 schrieb Konrad Weihmann: >> >> >> >> >> >> On 05.10.21 21:17, Alexander Kanavin wrote: >> >>> On Tue, 5 Oct 2021 at 19:44, Stefan Herbrechtsmeier >> >>> <[email protected] >> >>> <mailto:[email protected]>> wrote: >> >>> >> >>> >> >>> > A layer with thousands of recipes seems totally intractable. >> >>> >> >>> What is your concern? The high number of dependencies or to >> >>> handle it >> >>> via OE? >> >>> >> >>> >> >>> Generating recipes with a tool means the tool will be custom and >> >>> non-standard, and chances of anyone outside of your company >> >>> understanding, using and contributing to it are rather small. There's >> >>> also the problem of needing multiple versions of the same thing for >> >>> different >> >>> consumer items, which oe doesn't easily support. The link Robert >> >>> provided already exposes some of the pain points with that approach. >> >>> >> >>> I tend to think the best way forward (or least painful at least) is >> >>> to gradually improve what there already is, which is the npm class >> >>> and devtool, and send patches to various upstream njs projects when >> >>> they're using outdated dependencies or otherwise need changing. >> >>> >> >>> Alex >> >> >> >> Just to chime in :-), I like to question this approach of having >> >> multiple versions of the same in an image. >> >> As already outlined npm is horrible in many ways, but using the >> >> lockfile approach multiplies that even more. >> >> But I tend to agree that using the currently available oe-core code >> >> would be suited best for a broader audience >> > >> > But this audience loose the some basic features of OE (ex. version and >> > CVE check) and have to manual fix or ignore the licenses. >> > >> >> - in your own layer you simply can do whatever you like. >> > >> > But this make it impossible to share recipes. Why don't we create a >> > meta-nodejs with a different approach to avoid doing the same thing >> > multiple times. >> >> We could, but we have to think about the costs and ways to properly >> maintain it - and that seems to be the main reason. >> You could come up with a repo of your own and see who is interested in >> joining the development (as I for instance did with a repo for ruby >> gems) - still this would be hosted and maintained outside of core I >> think, due to the limited time and resources available. >> >> And **most important** the repo has to provide its own means of >> testing/CI. >> >> BTW I remember I had a talk with someone last year about it and putting >> a bunch of angular based apps into an image, with all npms as separate >> recipes resulted in ~100k of files. >> > We have discussed the severe impact on parsing time with 100k recipes. I personally worked on a node.js project with that many dependencies (but not an OE/Yocto deployment). Parsing time is already a pain point for 1000s of recipes. > >> so CI and testing has to be extra beefy to make it work - which maybe >> requires a good portion of automation, which then brings me back to >> Alex's suggestion to provide some patches to devtool, if there is a need >> for that. >> >> > >> >> While personally I think in the long run, every npm dependency has to >> >> be provided as a recipe of its own (even I know the costs of that >> >> pretty well)... esp when CVE checking and basic packaging hygiene >> >> should be enforced. >> > >> > Why should OE provide a solution with doesn't support this? I think >> this >> > is a main feature of OE. >> >> It actually does, but having this lockfile based approach, basically >> crams up all into a single recipe, which makes it a bit harder to check >> and whitelist any false positives or n.a. items, as you might have to do >> it for more than one recipe - but yeah, the basic CVE checking is part >> of core and works as expected >> >> > >> >> Nevertheless for oe-core I wouldn't want to change a lot right now, as >> >> there is simply a valid set of consumers missing that could enable us >> >> to do some proper testing. But yeah fixes/improvements for devtool are >> >> very welcome (also the available docu might need a touch) >> > >> > My problem is that the current approach doesn't support the build of an >> > express or angular application. Does it makes sense to add this feature >> > if we know that some basic OE features are missing (CVE and version >> > check) and fixing bugs is a nightmare? >> >> Just my personal opinion would be, to open up a layer on your own and >> let it grow and mature first before getting it in into core - but >> patches to devtool are very welcome >> > >> > Regards >> > Stefan >> > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#156743): https://lists.openembedded.org/g/openembedded-core/message/156743 Mute This Topic: https://lists.openembedded.org/mt/86089523/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
