On 2022-01-14 07:18, Alexander Kanavin wrote:
If we do seriously embark on making npm/go better, the first step could
be to make npm/go write out a reproducible manifest for the licenses and
sub-packages that can be verified against two recipe checksums during
fetch, and ensures no further network access is necessary. That alone
would make it a viable fetcher. Those manifests could contain all needed
information for further processing (e.g. versions, and what depends on
what etc.) And yes, it's a bundled self-contained approach, but that
matches how the rest of the world is using npm.
I can't speak to npm but for go this was where I wanted to see things
go. Just as work was done to avoid unexpected downloads of Python eggs I
always felt the key to improving go integration was some for of
automated SRC_URI generation. Once this would be available it could be
leveraged for licensing and such.
Stefan, by the way the reason (a) is not possible is that multiple go
applications can use a shared 'library' but different versions (or even
different git commit ids). The number of recipes and versions of recipes
required to support go applications quickly becomes difficult to manage.
Sorry, I should have included this in my first email.
MarkA
Alex
On Fri, 14 Jan 2022 at 12:43, Konrad Weihmann <[email protected]
<mailto:[email protected]>> wrote:
Guess the 3rd possibility is what's most likely - unfortunately there
doesn't seem much interest upstream for the way oe builds things, so it
might be fight against windmills.
option a) could be doable in the long run (but would at least require
upstream to acknowledge the oe way of doing things)
the more I think about it the more option b becomes likely, as there
isn't actually a real world consumer of any of these in core - so all
the needed quality control remains somewhere else.
for the tooling part, I think we need to enable devtool to create
recipes and source definitions separately, even for a whole dependency
tree - but I admit I have no idea if that is doable, as the devtool
sources has grown over the time into something very hard to read and
refactor (and if you would ask me, the above mentioned idea sounds like
a full rewrite of devtool, more or less from scratch).
(I remember there was an exchange about this idea in late December on
some of the lists)
my idea would be to
- drop npm/go support and move it to another place, one where there's
actually a consumer of them
- get in contact with upstream nodejs/go to discuss ways to build
things
with the project's definition of reproducibility
- start a devtool refactor enabling the creation of multiple
source/recipe definitions out of a dependency tree
- refactor go and npm support before moving them back into core
- add real life consumers to core to avoid future regressions
No idea what that would mean in terms of work and if that is even doable
On 14.01.22 12:16, Alexander Kanavin wrote:
> Three possible solutions, please:
>
> c) improve npm and go tooling in collaboration with respective
upstreams
> so that it fulfils our use cases.
>
> Both a and b are not tenable in my opinion.
>
> Alex
>
> On Fri, 14 Jan 2022 at 11:09, Stefan Herbrechtsmeier
> <[email protected]
<mailto:[email protected]>
> <mailto:[email protected]
<mailto:[email protected]>>> wrote:
>
> Hi,
>
> the npm and go integration doesn’t support a lot of common OE
> feature like:
> * Download proxy
> * Minimize image size (packet split, single copy, dead code
removal, …)
> * Software version management
> * Dependency management
> * License compliance
> * Vulnerability scanner
> * SBOM generator
>
> Even the `Download proxy` is only partly supported. The npm
packages
> could download artifacts during compile and Go projects
without vendor
> directory download dependencies during compile.
>
> The current state of npm and Go in OE aren’t complete, and a
user need
> to setup a DevOps chain outside of OE to take over the
missing parts.
> Furthermore, the DevOps chain needs its own download proxy,
and npm and
> Go supports cross compile by itself, so the advantage of the OE
> integration is minimal.
>
> Based on my work on a npm improvement in the last months I
see two
> possible solutions:
> a) Handle npm and Go projects like C/C++ or Python projects and
> create a
> recipe per project.
> b) Remove npm and Go support from OE and build artifacts via
external
> DevOps chain.
>
> I think the best solution would be a) because it avoids user
specific
> solution and allows collaboration. A solution between a) and
b) isn’t
> reasonable because it doesn’t solve the problem of an
additional DevOps
> chain and introduce a two-class society for languages.
>
> Does somebody use npm and Go and cares about the missing
features?
>
> Any feedback, opinions or interests would be helpful.
>
> Regards
> Stefan
>
>
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#1416):
https://lists.openembedded.org/g/openembedded-architecture/message/1416
Mute This Topic: https://lists.openembedded.org/mt/88417908/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
