Howdy! Thanks for sharing, a few comments inline.
Am Fr., 12. Feb. 2021 um 11:03 Uhr schrieb Simon Vogl <[email protected]>: > > I have some remarks and questions about the npm/nodejs support in dunfell > that I wanted to share. We are creating nodejs-based IoT edge solutions and > upgrading our build environments to Dunfell one by one. In the course of > this, we are switching to the new npm-implementation and found a few small > issues. > > Firstly, the do_configure() task takes quite some time to complete. After a > quick analysis, I saw that most of the time is being spent in creating the > npmrc files while packing the dependent packages. I wrote a small workaround > to directly create the file instead of calling 'npm config', which results in > a 3x-4x speedup: > > Signed-off-by: Simon Vogl <[email protected]> > --- > lib/bb/fetch2/npm.py | 9 +++++++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > > diff --git a/lib/bb/fetch2/npm.py b/lib/bb/fetch2/npm.py > index 4789850..2720d87 100644 > --- a/lib/bb/fetch2/npm.py > +++ b/lib/bb/fetch2/npm.py > @@ -97,13 +97,18 @@ class NpmEnvironment(object): > cmd = "NPM_CONFIG_GLOBALCONFIG=%s " % cfgfile + cmd > return runfetchcmd(cmd, d, workdir=workdir) > > + cfg = open(cfgfile, "a") > if self.configs: > for key, value in self.configs: > - _run("npm config set %s %s" % (key, shlex.quote(value))) > + cfg.write("%s=%s\n" % (key, shlex.quote(value))) > + #_run("npm config set %s %s" % (key, shlex.quote(value))) > > if configs: > for key, value in configs: > - _run("npm config set %s %s" % (key, shlex.quote(value))) > + cfg.write("%s=%s\n" % (key, shlex.quote(value))) > + # _run("npm config set %s %s" % (key, > shlex.quote(value))) > + > + cfg.close() > > if args: > for key, value in args: > -- > 2.7.4 > > Are there any side effects that I did not stumble over yet? And I'd LOVE to > have these calls running in a thread-pool for better performance... The main side effect is that you're effectively patching poky, which is bad for maintenance. > Secondly, our projects are based on typescript, so a native compile step is > necessary to create a compiled version for packing. We experimented with a > separate release branch to check in compiled versions, but this is not easy > to handle. I played around with npm.bbclass and found a way to extend > configure (!) with a call to our build script before packaging: > > diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass > index 068032a1e5..31535098cf 100644 > --- a/meta/classes/npm.bbclass > +++ b/meta/classes/npm.bbclass > @@ -170,6 +170,11 @@ python npm_do_configure() { > > # Configure the main package > with tempfile.TemporaryDirectory() as tmpdir: > + # install all (native) build dependencies, overrides npm cache: > + ret = os.system("npm i") > + # run build step: > + env.run("npm run build", args=[], workdir=d.getVar("S")) > + > tarball = npm_pack(env, d.getVar("S"), tmpdir) > npm_unpack(tarball, d.getVar("NPM_PACKAGE"), d) > > As we have plain JS packages as well, I put the modified configure() in a > subclass and this works for us, but it does not look like a clean solution to > me. How do you other IoT'ers address this situation? Again, patching poky is a bad idea. Creating custom bbclasses is much neater: you could create a base include, and pull that together with npm.bbclass into two final bbclasses of yours, like npm-js-voxel and npm-ts-voxel. The former would not have the compilation step. And, putting the typescript/webpack invocation into configure is also not exactly how things are meant to work. I know that the dependency tracking of npm is not easily compatible with bitbake, but the aim should be to 1) have a recipe that provides typescript-native 2) DEPEND on typescript-native in the recipe which you need to compile 3) add a do_compile stage that does the work. Greetz
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#52299): https://lists.yoctoproject.org/g/yocto/message/52299 Mute This Topic: https://lists.yoctoproject.org/mt/80579992/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
