If we do seriously embark on making npm/go better, the first step could be to make npm/go write out a reproducible manifest for the licenses and sub-packages that can be verified against two recipe checksums during fetch, and ensures no further network access is necessary. That alone would make it a viable fetcher. Those manifests could contain all needed information for further processing (e.g. versions, and what depends on what etc.) And yes, it's a bundled self-contained approach, but that matches how the rest of the world is using npm.
Alex On Fri, 14 Jan 2022 at 12:43, Konrad Weihmann <[email protected]> wrote: > Guess the 3rd possibility is what's most likely - unfortunately there > doesn't seem much interest upstream for the way oe builds things, so it > might be fight against windmills. > > option a) could be doable in the long run (but would at least require > upstream to acknowledge the oe way of doing things) > > the more I think about it the more option b becomes likely, as there > isn't actually a real world consumer of any of these in core - so all > the needed quality control remains somewhere else. > > for the tooling part, I think we need to enable devtool to create > recipes and source definitions separately, even for a whole dependency > tree - but I admit I have no idea if that is doable, as the devtool > sources has grown over the time into something very hard to read and > refactor (and if you would ask me, the above mentioned idea sounds like > a full rewrite of devtool, more or less from scratch). > (I remember there was an exchange about this idea in late December on > some of the lists) > > my idea would be to > > - drop npm/go support and move it to another place, one where there's > actually a consumer of them > - get in contact with upstream nodejs/go to discuss ways to build things > with the project's definition of reproducibility > - start a devtool refactor enabling the creation of multiple > source/recipe definitions out of a dependency tree > - refactor go and npm support before moving them back into core > - add real life consumers to core to avoid future regressions > > No idea what that would mean in terms of work and if that is even doable > > On 14.01.22 12:16, Alexander Kanavin wrote: > > Three possible solutions, please: > > > > c) improve npm and go tooling in collaboration with respective upstreams > > so that it fulfils our use cases. > > > > Both a and b are not tenable in my opinion. > > > > Alex > > > > On Fri, 14 Jan 2022 at 11:09, Stefan Herbrechtsmeier > > <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi, > > > > the npm and go integration doesn’t support a lot of common OE > > feature like: > > * Download proxy > > * Minimize image size (packet split, single copy, dead code removal, > …) > > * Software version management > > * Dependency management > > * License compliance > > * Vulnerability scanner > > * SBOM generator > > > > Even the `Download proxy` is only partly supported. The npm packages > > could download artifacts during compile and Go projects without > vendor > > directory download dependencies during compile. > > > > The current state of npm and Go in OE aren’t complete, and a user > need > > to setup a DevOps chain outside of OE to take over the missing parts. > > Furthermore, the DevOps chain needs its own download proxy, and npm > and > > Go supports cross compile by itself, so the advantage of the OE > > integration is minimal. > > > > Based on my work on a npm improvement in the last months I see two > > possible solutions: > > a) Handle npm and Go projects like C/C++ or Python projects and > > create a > > recipe per project. > > b) Remove npm and Go support from OE and build artifacts via external > > DevOps chain. > > > > I think the best solution would be a) because it avoids user specific > > solution and allows collaboration. A solution between a) and b) isn’t > > reasonable because it doesn’t solve the problem of an additional > DevOps > > chain and introduce a two-class society for languages. > > > > Does somebody use npm and Go and cares about the missing features? > > > > Any feedback, opinions or interests would be helpful. > > > > Regards > > Stefan > > > > > > > > > > > > > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1412): https://lists.openembedded.org/g/openembedded-architecture/message/1412 Mute This Topic: https://lists.openembedded.org/mt/88417908/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
