On Tue, Feb 04, 2020 at 08:53:59AM +0100, Ulrich Ölmann wrote:
> From: Björn Esser <b.es...@pengutronix.de>
> 
> This is a dummy package to specify and install required Node.js packages for 
> the
> target system.
> 
> The management of those packages and their dependencies is handled with
> Yarn (run through host-nodejs) to maintain a reproducible offline cache that 
> is
> stored in local_src/nodejs_packages.
> 
> Signed-off-by: Björn Esser <b.es...@pengutronix.de>
> [uol: adjusted commit message and Kconfig menu file's help texts, integrated
> Kconfig symbol NODEJS_PACKAGES_LICENSE, fixed NODEJS_PACKAGES_URL, removed one
> square bracket in a shell test and wrapped install/cp into call of execute and
> world/execute]
> Signed-off-by: Ulrich Ölmann <u.oelm...@pengutronix.de>
> ---
> v3 --> v4: adjusted commit message and Kconfig menu file's help texts & fixed
>            NODEJS_PACKAGES_URL
> Range-diff of v5 against v4:
> 3:  ccd2b44a8d92 ! 3:  1a73ebbd6e7e nodejs_packages: new package
>     @@ Commit message
>          stored in local_src/nodejs_packages.
> 
>          Signed-off-by: Björn Esser <b.es...@pengutronix.de>
>     -    [uol: adjusted commit message and Kconfig menu file's help texts & 
> fixed
>     -    NODEJS_PACKAGES_URL]
>     +    [uol: adjusted commit message and Kconfig menu file's help texts, 
> integrated
>     +    Kconfig symbol NODEJS_PACKAGES_LICENSE, fixed NODEJS_PACKAGES_URL, 
> removed one
>     +    square bracket in a shell test and wrapped install/cp into call of 
> execute and
>     +    world/execute]
>          Signed-off-by: Ulrich Ölmann <u.oelm...@pengutronix.de>
> 
>       ## rules/nodejs_packages.in (new) ##
>     @@ rules/nodejs_packages.in (new)
>      +
>      +          [1] https://legacy.yarnpkg.com/en/docs/dependency-versions/
>      +
>     ++config NODEJS_PACKAGES_LICENSE
>     ++        string
>     ++        prompt "Licenses of to be installed Node.js packages"
>     ++        default "unknown"
>     ++        help
>     ++          Manually compiled list of licenses that the to be installed 
> Node.js
>     ++          packages are distributed under, see description of the 
> variable
>     ++          *_LICENSE in [1].
>     ++
>     ++          ATTENTION: be aware of the fact that Node.js packages usually 
> depend
>     ++          on additional Node.js packages that Yarn installs as well and 
> that
>     ++          this propagates recursively resulting in an unknown number of 
> Node.js
>     ++          packages on the target with an unknown set of licenses that 
> they may
>     ++          be utilized under. Hence you carefully need to examine all of 
> them
>     ++          individually to create a reliable list of licenses.
>     ++
>     ++          [1] 
> https://www.ptxdist.org/doc/dev_manual.html#some-notes-about-licenses
>     ++
>      +config NODEJS_PACKAGES_OFFLINE
>      +        bool
>     -+        prompt "lock offline cache for offline/production use"
>     ++        prompt "Lock offline cache for offline/production use"
>      +        help
>     -+          This option locks the offline cache for installing the nodejs 
> packages
>     -+          and allows their installation during the build of the BSP 
> without any
>     -+          need for internet connectivity.
>     ++          This option locks the offline cache for installing the Node.js
>     ++          packages and allows their installation during the build of 
> the BSP
>     ++          without any need for internet connectivity.
>      +
>      +          Enabling this option is highly recommended for production use.
>      +
>     @@ rules/nodejs_packages.make (new)
>      +NODEJS_PACKAGES_URL     := 
> lndir://$(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)
>      +NODEJS_PACKAGES_DIR     := $(BUILDDIR)/$(NODEJS_PACKAGES)
>      +NODEJS_PACKAGES_CACHE   := 
> $(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)/yarn_cache
>     -+NODEJS_PACKAGES_LICENSE := unknown
>     ++NODEJS_PACKAGES_LICENSE := $(call remove_quotes, 
> $(PTXCONF_NODEJS_PACKAGES_LICENSE))
>      +
>      +NODEJS_PACKAGES_LIST    := $(call remove_quotes, 
> $(PTXCONF_NODEJS_PACKAGES_LIST))
>      +
>     @@ rules/nodejs_packages.make (new)
>      +                \"license\": \"UNLICENSED\", \
>      +                \"private\": true \
>      +        }" > $(NODEJS_PACKAGES_DIR)/package.json
>     -+        if [[ ! -f $(YARN_LOCK) ]]; then \
>     ++        if [ ! -f $(YARN_LOCK) ]; then \
>      +                touch $(YARN_LOCK); \
>      +                ln -fs $(YARN_LOCK) $(NODEJS_PACKAGES_DIR)/yarn.lock; \
>      +        fi
>     @@ rules/nodejs_packages.make (new)
>      +
>      +$(STATEDIR)/nodejs_packages.install:
>      +        @$(call targetinfo)
>     -+        install -dm 0755 $(NODEJS_PACKAGES_PKGDIR)/usr/lib
>     -+        cp -pr $(NODEJS_PACKAGES_DIR)/node_modules 
> $(NODEJS_PACKAGES_PKGDIR)/usr/lib
>     ++        @$(call world/execute, HOST_UA_NODESET, \
>     ++                install -dm 0755 $(NODEJS_PACKAGES_PKGDIR)/usr/lib)
>     ++        @$(call execute, HOST_UA_NODESET, \
>     ++                cp -pr $(NODEJS_PACKAGES_DIR)/node_modules \
>     ++                        $(NODEJS_PACKAGES_PKGDIR)/usr/lib)
>      +        @$(call touch)
>      +
>      +# 
> ----------------------------------------------------------------------------
> 
>  rules/nodejs_packages.in   | 73 ++++++++++++++++++++++++++++++
>  rules/nodejs_packages.make | 93 ++++++++++++++++++++++++++++++++++++++
>  2 files changed, 166 insertions(+)
>  create mode 100644 rules/nodejs_packages.in
>  create mode 100644 rules/nodejs_packages.make
> 
> diff --git a/rules/nodejs_packages.in b/rules/nodejs_packages.in
> new file mode 100644
> index 000000000000..aea44205da41
> --- /dev/null
> +++ b/rules/nodejs_packages.in
> @@ -0,0 +1,73 @@
> +## SECTION=bytecode_engines
> +
> +menuconfig NODEJS_PACKAGES
> +     tristate
> +     select HOST_YARN
> +     select NODEJS
> +     prompt "Node.js packages (target)     "
> +     help
> +       Reproducibly install Node.js packages that will be available
> +       system-wide on the target.
> +
> +       You MUST specify at least one Node.js package in the submenu that is
> +       shown after enabling this option.
> +
> +
> +if NODEJS_PACKAGES
> +
> +config NODEJS_PACKAGES_LIST
> +     string
> +     prompt "Node.js packages to install"
> +     help
> +       A white-space separated list of Node.js packages including their
> +       versions (e.g. express@4.17.x or socket.io@'>=2.1.0 <=2.3.0'). NPM
> +       semvers are fully supported, see [1].
> +
> +       The packages are managed with Yarn and are guaranteed to be
> +       reproducible and immutable as long as the download location stays
> +       accessible or the offline cache is not altered.
> +
> +       If the list of Node.js packages to be installed has been populated for
> +       the first time or if it has been modified later on one needs to
> +       (re-)build this package to download the desired Node.js packages and
> +       update the offline cache. Afterwards it is highly recommended to
> +       commit the path '${PTXDIST_WORKSPACE}/local_src/nodejs_packages/' into
> +       the version control system and to lock the package cache.
> +
> +       As a prerequisite the path '${PTXDIST_WORKSPACE}/local_src/nodejs_
> +       packages/' must exist in the BSP *before* building this package.
> +
> +       [1] https://legacy.yarnpkg.com/en/docs/dependency-versions/
> +
> +config NODEJS_PACKAGES_LICENSE
> +     string
> +     prompt "Licenses of to be installed Node.js packages"
> +     default "unknown"
> +     help
> +       Manually compiled list of licenses that the to be installed Node.js
> +       packages are distributed under, see description of the variable
> +       *_LICENSE in [1].
> +
> +       ATTENTION: be aware of the fact that Node.js packages usually depend
> +       on additional Node.js packages that Yarn installs as well and that
> +       this propagates recursively resulting in an unknown number of Node.js
> +       packages on the target with an unknown set of licenses that they may
> +       be utilized under. Hence you carefully need to examine all of them
> +       individually to create a reliable list of licenses.
> +
> +       [1] 
> https://www.ptxdist.org/doc/dev_manual.html#some-notes-about-licenses
> +
> +config NODEJS_PACKAGES_OFFLINE
> +     bool
> +     prompt "Lock offline cache for offline/production use"
> +     help
> +       This option locks the offline cache for installing the Node.js
> +       packages and allows their installation during the build of the BSP
> +       without any need for internet connectivity.
> +
> +       Enabling this option is highly recommended for production use.
> +
> +       Do NOT forget to commit the path '${PTXDIST_WORKSPACE}/local_src/
> +       nodejs_packages/' into the BSP's version control system!
> +
> +endif
> diff --git a/rules/nodejs_packages.make b/rules/nodejs_packages.make
> new file mode 100644
> index 000000000000..cdde30f38b0c
> --- /dev/null
> +++ b/rules/nodejs_packages.make
> @@ -0,0 +1,93 @@
> +# -*-makefile-*-
> +#
> +# Copyright (C) 2020 by Bjoern Esser <b...@pengutronix.de>
> +#
> +# For further information about the PTXdist project and license conditions
> +# see the README file.
> +#
> +
> +#
> +# We provide this package
> +#
> +PACKAGES-$(PTXCONF_NODEJS_PACKAGES) += nodejs_packages
> +
> +#
> +# Paths and names
> +#
> +NODEJS_PACKAGES_VERSION      := 0.0.1
> +NODEJS_PACKAGES              := nodejs_packages-$(NODEJS_PACKAGES_VERSION)
> +NODEJS_PACKAGES_LOCAL        := local_src/nodejs_packages
> +NODEJS_PACKAGES_URL  := lndir://$(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)
> +NODEJS_PACKAGES_DIR  := $(BUILDDIR)/$(NODEJS_PACKAGES)
> +NODEJS_PACKAGES_CACHE        := 
> $(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)/yarn_cache
> +NODEJS_PACKAGES_LICENSE      := $(call remove_quotes, 
> $(PTXCONF_NODEJS_PACKAGES_LICENSE))
> +
> +NODEJS_PACKAGES_LIST := $(call remove_quotes, 
> $(PTXCONF_NODEJS_PACKAGES_LIST))
> +
> +YARN_LOCK    := $(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)/yarn.lock
> +YARN_OPTS    := \
> +     --cwd "$(NODEJS_PACKAGES_DIR)" \
> +     --cache-folder "$(NODEJS_PACKAGES_CACHE)" \
> +     --link-duplicates \
> +     --production=true
> +
> +ifdef PTXCONF_NODEJS_PACKAGES_OFFLINE
> +YARN_OPTS    += \
> +     --frozen-lockfile \
> +     --offline
> +endif
> +
> +# 
> ----------------------------------------------------------------------------
> +# Compile
> +# 
> ----------------------------------------------------------------------------
> +
> +$(STATEDIR)/nodejs_packages.compile:
> +     @$(call targetinfo)
> +     mkdir -p $(NODEJS_PACKAGES_CACHE)
> +     printf "{ \
> +             \"name\": \"nodejs_packages\", \
> +             \"version\": \"$(NODEJS_PACKAGES_VERSION)\", \
> +             \"license\": \"UNLICENSED\", \
> +             \"private\": true \
> +     }" > $(NODEJS_PACKAGES_DIR)/package.json
> +     if [ ! -f $(YARN_LOCK) ]; then \
> +             touch $(YARN_LOCK); \
> +             ln -fs $(YARN_LOCK) $(NODEJS_PACKAGES_DIR)/yarn.lock; \
> +     fi

I think we should fail here if NODEJS_PACKAGES_OFFLINE is set and the lock
file is missing. Or will yarn produce a sensible error message in this
case?

> +     yarn $(YARN_OPTS) add $(NODEJS_PACKAGES_LIST)
> +     find $(NODEJS_PACKAGES_CACHE) -type f -name '.yarn-tarball.tgz' -delete
> +     @$(call touch)
> +
> +# 
> ----------------------------------------------------------------------------
> +# Install
> +# 
> ----------------------------------------------------------------------------
> +
> +$(STATEDIR)/nodejs_packages.install:
> +     @$(call targetinfo)
> +     @$(call world/execute, HOST_UA_NODESET, \
> +             install -dm 0755 $(NODEJS_PACKAGES_PKGDIR)/usr/lib)

add -v to make it visible at runtime.

> +     @$(call execute, HOST_UA_NODESET, \
> +             cp -pr $(NODEJS_PACKAGES_DIR)/node_modules \
> +                     $(NODEJS_PACKAGES_PKGDIR)/usr/lib)

Here as well.

Michael

> +     @$(call touch)
> +
> +# 
> ----------------------------------------------------------------------------
> +# Target-Install
> +# 
> ----------------------------------------------------------------------------
> +
> +$(STATEDIR)/nodejs_packages.targetinstall:
> +     @$(call targetinfo)
> +
> +     @$(call install_init, nodejs_packages)
> +     @$(call install_fixup, nodejs_packages,PRIORITY,optional)
> +     @$(call install_fixup, nodejs_packages,SECTION,base)
> +     @$(call install_fixup, nodejs_packages,AUTHOR,"Bjoern Esser 
> <b...@pengutronix.de>")
> +     @$(call install_fixup, nodejs_packages,DESCRIPTION,missing)
> +
> +     $(call install_tree, nodejs_packages, 0, 0, -, /usr/lib/node_modules/)
> +
> +     @$(call install_finish, nodejs_packages)
> +
> +     @$(call touch)
> +
> +# vim: syntax=make
> -- 
> 2.25.0
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Reply via email to