Re: Proposal: Reboot nodejs library/modules in Fedora
Hi! I'm one of the maintainers of Grafana in Fedora. > Not that the tests would go away, but that the build/test dependencies would > be bundled in a tarball. I really like the idea, however (using latest Grafana): $ du -sch node_modules 1,7Gnode_modules Some of the largest dependencies: $ du -am node_modules | sort -n -r | head -n 20 1719node_modules 291 node_modules/@storybook 85 node_modules/@babel 81 node_modules/@storybook/core 78 node_modules/@storybook/core/node_modules 73 node_modules/monaco-editor 66 node_modules/@types 58 node_modules/@microsoft 54 node_modules/@microsoft/api-extractor 53 node_modules/typescript 52 node_modules/@microsoft/api-extractor/node_modules 50 node_modules/typescript/lib 49 node_modules/@microsoft/api-extractor/node_modules/typescript 46 node_modules/@microsoft/api-extractor/node_modules/typescript/lib 38 node_modules/@svgr 36 node_modules/@svgr/webpack/node_modules 36 node_modules/@svgr/webpack 35 node_modules/@svgr/webpack/node_modules/@babel 33 node_modules/monaco-editor/dev/vs 33 node_modules/monaco-editor/dev $ ls node_modules | wc -l 2017 Compressing the node_modules folder using zstd: default level: 179 MB zstd level 19: 108 MB How can/should we bundle that? Is this size acceptable in a SRPM? Bundling it would allow us to build the frontend and run the frontend tests as part of the spec, and we can apply patches properly (without (re)generating anything locally). Most of the patches at least, in case some security patch requires an upgrade of a npm dependency, we also need to recreate the vendor tarball. Cheers, Andreas ___ nodejs mailing list -- nodejs@lists.fedoraproject.org To unsubscribe send an email to nodejs-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/nodejs@lists.fedoraproject.org
Re: Proposal: Reboot nodejs library/modules in Fedora
On Thu, Aug 27, 2020 at 3:11 PM Troy Dawson wrote:
>
> On Thu, Jul 30, 2020 at 1:15 PM Troy Dawson wrote:
> >
> > On Wed, Jul 22, 2020 at 6:31 AM Ben Rosser wrote:
> > >
> > > On Tue, Jul 21, 2020 at 10:56 AM Tom Hughes via nodejs
> > > wrote:
> > > >
> > > > On 21/07/2020 15:39, Troy Dawson wrote:
> > > >
> > > > > Problem2: Many of the bundled libraries are archful, meaning they need
> > > > > to be compiled. What if an essential nodejs library was originally a
> > > > > noarch?
> > > > > Solution2: All bundled nodejs library packages will be treached as
> > > > > archful, meaning they will need to be built on each supported arch.
> > > >
> > > > I think "many" is stretching things.
> > > >
> > > > There were never more than about a dozen or so archful modules in
> > > > Fedora and most of them were not widely used.
> > > >
> > > > I suspect that archful modules are probably best kept packaged as
> > > > they are now mostly just because trying to build them as part of
> > > > a large bundled tree is going to be incredibly painful.
> > >
> > > I think this makes sense as well. Essentially, we should just say that
> > > archful nodejs modules are "essential" (in the words of the proposal)
> > > and therefore need to be packaged and shipped separately.
> > >
> > > I think a "npm2rpm" script would have to do a couple things-- does
> > > this sound about right?
> > >
> > > a) recursively parse package.json to get a list of all dependencies
> > > and tarballs, and use this to write out a list of Sources.
> > > b) for each package, also look at package.json to see what files need
> > > to be installed, and write out instructions for the %install section.
> > > c) generate a "License" field for the package and also install all the
> > > relevant licenses in %license.
> > >
> > > There should probably be a way to specify that a given dependency is
> > > not being bundled too, to stop the parser from descending into that
> > > dependency tree. I imagine that (at least during the transition to
> > > this and maybe afterwards, in the case of archful or essential
> > > modules) it would be very useful to have packages with a mixture of
> > > bundled and unbundled dependencies.
> > >
> > > Ben Rosser
> > >
> >
> > Hi Ben,
> > Sorry it's taken so long for me to properly reply to this.
> > I'm going to refer to your last paragraph (having a mixed bundled and
> > unbundled) as (d)
> >
> > For section (a) and (b), would you rather write our own script, or use
> > npm or yarn ?
> >
> > In my mind I was thinking of doing something like "npm install" then
> > using the resulting node_modules directory.
> > Turn that into one tarball.
> >
> > It sounds like you would like a seperate source for each package, at
> > least for the top level package.
> > I can see some pro's and con's for both.
> >
> > Single Tarball - Good Things:
> > - Don't have to write a new script for downloading.
> > - Everything in one place.
> > - It's laid out how npm would lay it out.
> > Single Tarball - Bad Things:
> > - Harder to do mixed bundles (d)
> > - Have to redo the entire tarball for any update to any part of it.
> >
> > Multi Sources - Good Things:
> > - Have everything parsed and labeled at creation time.
> > - Easier to do "this is only for build/test" and remove them for runtime.
> > - Easier to do mixed bundles (d)
> > Multi Sources - Bad Things:
> > - Have to create our own script (?)
> > - Complicated "Sources" section in spec file (100's to 1000)
> >
> > I'm going to research this some more. But if you already have some
> > idea's, I'd love to hear them.
> >
> > Troy
>
> I've got a package that won't install anymore, and finally got some
> time, so I've been taking a swipe at a bundling script, see how it
> goes.
> This isn't a complete npm2rpm. I already have a spec for this rpm.
> But here is what I currently have
>
> PACKAGE="node-static"
> VERSION="$(npm view ${PACKAGE} version)"
> TMP_DIR=$(mktemp -d -t ci-XX)
> mkdir ${TMP_DIR}
> pushd ${TMP_DIR}
> npm pack ${PACKAGE}
> tar xfz *.tgz
> cd package
> npm install --no-optional --only=prod
> # FIND SOMEWHERE TO PUT LICENSES
> # FOR NOW JUST PRINT THEM OUT
> find . -name "package.json" -exec jq .license {} \; | sort -u
> mv node_modules/ node_modules_prod
> npm install --no-optional --only=dev
> mv node_modules/ node_modules_dev
> tar cfz ../${PACKAGE}-${VERSION}-nm-prod.tgz node_modules_prod
> tar cfz ../${PACKAGE}-${VERSION}-nm-dev.tgz node_modules_dev
> cd ..
> cp ${PACKAGE}-${VERSION}* $HOME/rpmbuild/SOURCES
> popd
> rm -rf ${TMP_DIR}
>
> This is a completely noarch package.
> What I have in my %install section is this.
>
> %install
> mkdir -p %{buildroot}%{nodejs_sitelib}/%{npm_name}
> cp -pr benchmark bin examples lib package.json test
> %{buildroot}%{nodejs_sitelib}/%{npm_name}
>
> # Setup Binaries
> mkdir %{buildroot}%{_bindir}
> ln -s %{nodejs_sitelib}/%{npm_name}/bin/cli.js %{buildroot}%{_bindir}/static
>
> # Setup node modules
> cd
Re: Proposal: Reboot nodejs library/modules in Fedora
On Thu, Jul 30, 2020 at 1:15 PM Troy Dawson wrote:
>
> On Wed, Jul 22, 2020 at 6:31 AM Ben Rosser wrote:
> >
> > On Tue, Jul 21, 2020 at 10:56 AM Tom Hughes via nodejs
> > wrote:
> > >
> > > On 21/07/2020 15:39, Troy Dawson wrote:
> > >
> > > > Problem2: Many of the bundled libraries are archful, meaning they need
> > > > to be compiled. What if an essential nodejs library was originally a
> > > > noarch?
> > > > Solution2: All bundled nodejs library packages will be treached as
> > > > archful, meaning they will need to be built on each supported arch.
> > >
> > > I think "many" is stretching things.
> > >
> > > There were never more than about a dozen or so archful modules in
> > > Fedora and most of them were not widely used.
> > >
> > > I suspect that archful modules are probably best kept packaged as
> > > they are now mostly just because trying to build them as part of
> > > a large bundled tree is going to be incredibly painful.
> >
> > I think this makes sense as well. Essentially, we should just say that
> > archful nodejs modules are "essential" (in the words of the proposal)
> > and therefore need to be packaged and shipped separately.
> >
> > I think a "npm2rpm" script would have to do a couple things-- does
> > this sound about right?
> >
> > a) recursively parse package.json to get a list of all dependencies
> > and tarballs, and use this to write out a list of Sources.
> > b) for each package, also look at package.json to see what files need
> > to be installed, and write out instructions for the %install section.
> > c) generate a "License" field for the package and also install all the
> > relevant licenses in %license.
> >
> > There should probably be a way to specify that a given dependency is
> > not being bundled too, to stop the parser from descending into that
> > dependency tree. I imagine that (at least during the transition to
> > this and maybe afterwards, in the case of archful or essential
> > modules) it would be very useful to have packages with a mixture of
> > bundled and unbundled dependencies.
> >
> > Ben Rosser
> >
>
> Hi Ben,
> Sorry it's taken so long for me to properly reply to this.
> I'm going to refer to your last paragraph (having a mixed bundled and
> unbundled) as (d)
>
> For section (a) and (b), would you rather write our own script, or use
> npm or yarn ?
>
> In my mind I was thinking of doing something like "npm install" then
> using the resulting node_modules directory.
> Turn that into one tarball.
>
> It sounds like you would like a seperate source for each package, at
> least for the top level package.
> I can see some pro's and con's for both.
>
> Single Tarball - Good Things:
> - Don't have to write a new script for downloading.
> - Everything in one place.
> - It's laid out how npm would lay it out.
> Single Tarball - Bad Things:
> - Harder to do mixed bundles (d)
> - Have to redo the entire tarball for any update to any part of it.
>
> Multi Sources - Good Things:
> - Have everything parsed and labeled at creation time.
> - Easier to do "this is only for build/test" and remove them for runtime.
> - Easier to do mixed bundles (d)
> Multi Sources - Bad Things:
> - Have to create our own script (?)
> - Complicated "Sources" section in spec file (100's to 1000)
>
> I'm going to research this some more. But if you already have some
> idea's, I'd love to hear them.
>
> Troy
I've got a package that won't install anymore, and finally got some
time, so I've been taking a swipe at a bundling script, see how it
goes.
This isn't a complete npm2rpm. I already have a spec for this rpm.
But here is what I currently have
PACKAGE="node-static"
VERSION="$(npm view ${PACKAGE} version)"
TMP_DIR=$(mktemp -d -t ci-XX)
mkdir ${TMP_DIR}
pushd ${TMP_DIR}
npm pack ${PACKAGE}
tar xfz *.tgz
cd package
npm install --no-optional --only=prod
# FIND SOMEWHERE TO PUT LICENSES
# FOR NOW JUST PRINT THEM OUT
find . -name "package.json" -exec jq .license {} \; | sort -u
mv node_modules/ node_modules_prod
npm install --no-optional --only=dev
mv node_modules/ node_modules_dev
tar cfz ../${PACKAGE}-${VERSION}-nm-prod.tgz node_modules_prod
tar cfz ../${PACKAGE}-${VERSION}-nm-dev.tgz node_modules_dev
cd ..
cp ${PACKAGE}-${VERSION}* $HOME/rpmbuild/SOURCES
popd
rm -rf ${TMP_DIR}
This is a completely noarch package.
What I have in my %install section is this.
%install
mkdir -p %{buildroot}%{nodejs_sitelib}/%{npm_name}
cp -pr benchmark bin examples lib package.json test
%{buildroot}%{nodejs_sitelib}/%{npm_name}
# Setup Binaries
mkdir %{buildroot}%{_bindir}
ln -s %{nodejs_sitelib}/%{npm_name}/bin/cli.js %{buildroot}%{_bindir}/static
# Setup node modules
cd %{buildroot}%{nodejs_sitelib}/%{npm_name}
tar xfz %{SOURCE1}
mkdir -p node_modules
cd node_modules
ln -s ../node_modules_prod/* .
ln -s ../node_modules_prod/.bin .
This builds just fine, which is great, except for one thing.
The current rpm macros file that is figuring out the bundled
dependencies, is only looking to see if
Re: Proposal: Reboot nodejs library/modules in Fedora
On Wed, Jul 22, 2020 at 6:31 AM Ben Rosser wrote: > > On Tue, Jul 21, 2020 at 10:56 AM Tom Hughes via nodejs > wrote: > > > > On 21/07/2020 15:39, Troy Dawson wrote: > > > > > Problem2: Many of the bundled libraries are archful, meaning they need > > > to be compiled. What if an essential nodejs library was originally a > > > noarch? > > > Solution2: All bundled nodejs library packages will be treached as > > > archful, meaning they will need to be built on each supported arch. > > > > I think "many" is stretching things. > > > > There were never more than about a dozen or so archful modules in > > Fedora and most of them were not widely used. > > > > I suspect that archful modules are probably best kept packaged as > > they are now mostly just because trying to build them as part of > > a large bundled tree is going to be incredibly painful. > > I think this makes sense as well. Essentially, we should just say that > archful nodejs modules are "essential" (in the words of the proposal) > and therefore need to be packaged and shipped separately. > > I think a "npm2rpm" script would have to do a couple things-- does > this sound about right? > > a) recursively parse package.json to get a list of all dependencies > and tarballs, and use this to write out a list of Sources. > b) for each package, also look at package.json to see what files need > to be installed, and write out instructions for the %install section. > c) generate a "License" field for the package and also install all the > relevant licenses in %license. > > There should probably be a way to specify that a given dependency is > not being bundled too, to stop the parser from descending into that > dependency tree. I imagine that (at least during the transition to > this and maybe afterwards, in the case of archful or essential > modules) it would be very useful to have packages with a mixture of > bundled and unbundled dependencies. > > Ben Rosser > Hi Ben, Sorry it's taken so long for me to properly reply to this. I'm going to refer to your last paragraph (having a mixed bundled and unbundled) as (d) For section (a) and (b), would you rather write our own script, or use npm or yarn ? In my mind I was thinking of doing something like "npm install" then using the resulting node_modules directory. Turn that into one tarball. It sounds like you would like a seperate source for each package, at least for the top level package. I can see some pro's and con's for both. Single Tarball - Good Things: - Don't have to write a new script for downloading. - Everything in one place. - It's laid out how npm would lay it out. Single Tarball - Bad Things: - Harder to do mixed bundles (d) - Have to redo the entire tarball for any update to any part of it. Multi Sources - Good Things: - Have everything parsed and labeled at creation time. - Easier to do "this is only for build/test" and remove them for runtime. - Easier to do mixed bundles (d) Multi Sources - Bad Things: - Have to create our own script (?) - Complicated "Sources" section in spec file (100's to 1000) I'm going to research this some more. But if you already have some idea's, I'd love to hear them. Troy ___ nodejs mailing list -- nodejs@lists.fedoraproject.org To unsubscribe send an email to nodejs-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/nodejs@lists.fedoraproject.org
Re: Proposal: Reboot nodejs library/modules in Fedora
On Wed, Jul 29, 2020 at 2:44 PM Tom Hughes wrote: > > On 29/07/2020 22:38, Troy Dawson wrote: > > > I wanted to get a list of essential nodejs packages, and I thought of > > getting a list of nodejs packages that are required by non-nodejs > > packages. > > How have you defined essential? > That's what I still don't know and I'd love some feedback. nodejs is one of the few languages in Fedora that allows bundling. It's fairly new (a year or two) and I'd like to push it to it's extreme. Ideal world (not gonna happen, but this is my 'everything works' goal) Fedora only has nodejs and npm as packages. If a package needs a nodejs library, they bundle it, and there is an easy way to bundle it. If a developer and/or end user needs a nodejs library, they use npm, and not an rpm. With that as the goal, my definition of essential is any nodejs library that so many other packages require, that it is more work to bundle it everywhere, versus having a Fedora package for it. I'm willing to to change that definition of essential, but that's what I'm starting off with. > > That list is amazingly small. > > > > # Binaries / Runtime packages > > statsd -> nodejs-generic-pool > > R-shiny -> nodejs-showdown > > notepadqq -> nodejs-shelljs > > mocha -> uglify-js and nodejs-jade > > kosmtik -> 11 nodejs libraries > > jake -> 5 nodejs libraries > > elasticdump -> 3 nodejs libraries > > discord-irc -> 10 nodejs libraries > > carto -> 4 nodejs libraries > > As both kosmtik and carto originate with me - indeed kosmtik > is kind of the original cause of my involvement in Node.js > pacaging I can comment a bit on those. > > I wouldn't actually worry too much about keeping kosmtik to > be honest. > > While carto is probably more useful the current version is > rather out of date, because of tricky dependency issues in > the newer versions so that dependency count will go up if it > is updated I think. > > I also wonder if you missed some build/test toolchains > that are required? Only that is where of the nasty trees > of dependencies are I think. > Actually, that is one of the points of this. Making those build/test dependency trees go away. Not that the tests would go away, but that the build/test dependencies would be bundled in a tarball. Each time you update your nodejs library, you run the script that re-makes the build-test tarball. During the build, they would be untarred, build and tests would be run, and they go away when the rpm is packaged. Troy ___ nodejs mailing list -- nodejs@lists.fedoraproject.org To unsubscribe send an email to nodejs-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/nodejs@lists.fedoraproject.org
Re: Proposal: Reboot nodejs library/modules in Fedora
On 29/07/2020 22:38, Troy Dawson wrote: I wanted to get a list of essential nodejs packages, and I thought of getting a list of nodejs packages that are required by non-nodejs packages. How have you defined essential? That list is amazingly small. # Binaries / Runtime packages statsd -> nodejs-generic-pool R-shiny -> nodejs-showdown notepadqq -> nodejs-shelljs mocha -> uglify-js and nodejs-jade kosmtik -> 11 nodejs libraries jake -> 5 nodejs libraries elasticdump -> 3 nodejs libraries discord-irc -> 10 nodejs libraries carto -> 4 nodejs libraries As both kosmtik and carto originate with me - indeed kosmtik is kind of the original cause of my involvement in Node.js pacaging I can comment a bit on those. I wouldn't actually worry too much about keeping kosmtik to be honest. While carto is probably more useful the current version is rather out of date, because of tricky dependency issues in the newer versions so that dependency count will go up if it is updated I think. I also wonder if you missed some build/test toolchains that are required? Only that is where of the nasty trees of dependencies are I think. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ nodejs mailing list -- nodejs@lists.fedoraproject.org To unsubscribe send an email to nodejs-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/nodejs@lists.fedoraproject.org
Re: Proposal: Reboot nodejs library/modules in Fedora
On Wed, Jul 22, 2020 at 8:24 AM Troy Dawson wrote: > > On Wed, Jul 22, 2020 at 6:31 AM Ben Rosser wrote: > > > > On Tue, Jul 21, 2020 at 10:56 AM Tom Hughes via nodejs > > wrote: > > > > > > On 21/07/2020 15:39, Troy Dawson wrote: > > > > > > > Problem2: Many of the bundled libraries are archful, meaning they need > > > > to be compiled. What if an essential nodejs library was originally a > > > > noarch? > > > > Solution2: All bundled nodejs library packages will be treached as > > > > archful, meaning they will need to be built on each supported arch. > > > > > > I think "many" is stretching things. > > > > > > There were never more than about a dozen or so archful modules in > > > Fedora and most of them were not widely used. > > > > > > I suspect that archful modules are probably best kept packaged as > > > they are now mostly just because trying to build them as part of > > > a large bundled tree is going to be incredibly painful. > > > > I think this makes sense as well. Essentially, we should just say that > > archful nodejs modules are "essential" (in the words of the proposal) > > and therefore need to be packaged and shipped separately. > > > > I think a "npm2rpm" script would have to do a couple things-- does > > this sound about right? > > > > a) recursively parse package.json to get a list of all dependencies > > and tarballs, and use this to write out a list of Sources. > > b) for each package, also look at package.json to see what files need > > to be installed, and write out instructions for the %install section. > > c) generate a "License" field for the package and also install all the > > relevant licenses in %license. > > > > There should probably be a way to specify that a given dependency is > > not being bundled too, to stop the parser from descending into that > > dependency tree. I imagine that (at least during the transition to > > this and maybe afterwards, in the case of archful or essential > > modules) it would be very useful to have packages with a mixture of > > bundled and unbundled dependencies. > > > > Ben Rosser > > > > You are both right. > Odd as it seems, I kept seeing nodejs-sqlite3 come up, which is archful. > I had never done a real search. > It turns out that the various packages I had been working on require > it, and only two other packages. > > So ya, Ben, I like you idea. I was looking at a very skewed view. > > Troy Sorry for being so quiet. I was going through all of Fedora's dependencies, and double checking my results because I was having a hard time believing what I was seeing. I wanted to get a list of essential nodejs packages, and I thought of getting a list of nodejs packages that are required by non-nodejs packages. That list is amazingly small. # Binaries / Runtime packages statsd -> nodejs-generic-pool R-shiny -> nodejs-showdown notepadqq -> nodejs-shelljs mocha -> uglify-js and nodejs-jade kosmtik -> 11 nodejs libraries jake -> 5 nodejs libraries elasticdump -> 3 nodejs libraries discord-irc -> 10 nodejs libraries carto -> 4 nodejs libraries # Source Dependencies statsd R-shiny python-webassets perl-Mojolicious-Plugin-AssetPack mocha kosmtik jake elasticdump discord-irc carto Note: Due to the current high volume of uninstallable packages, and being in the middle of a mass rebuild, it's possible I'm off by a few packages. I also left off js-jquery*. But even if you think we should include the js-* packages, that's still only about 5-10 more packages. Most of the binary/runtime dependencies did not have many duplicates. So we are looking at 30-40 packages if we wanted to consider all of them "essential". I would think it would be better to have the high volume ones (jake, kosmtik, discord-irc) bundle their nodejs dependencies. So, after this exercise, I'm sort of at a loss as to what we would consider essential. So I'm going to throw out a short list. Please add or subtract from it. Once we figure on a list, we can start figuring out a plan on how to trim Fedora down to that list. nodejs (Have to have) npm (Have to have) yarnpkg js-jquery uglify-js Thanks Troy ___ nodejs mailing list -- nodejs@lists.fedoraproject.org To unsubscribe send an email to nodejs-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/nodejs@lists.fedoraproject.org
Re: Proposal: Reboot nodejs library/modules in Fedora
On Wed, Jul 22, 2020 at 6:31 AM Ben Rosser wrote: > > On Tue, Jul 21, 2020 at 10:56 AM Tom Hughes via nodejs > wrote: > > > > On 21/07/2020 15:39, Troy Dawson wrote: > > > > > Problem2: Many of the bundled libraries are archful, meaning they need > > > to be compiled. What if an essential nodejs library was originally a > > > noarch? > > > Solution2: All bundled nodejs library packages will be treached as > > > archful, meaning they will need to be built on each supported arch. > > > > I think "many" is stretching things. > > > > There were never more than about a dozen or so archful modules in > > Fedora and most of them were not widely used. > > > > I suspect that archful modules are probably best kept packaged as > > they are now mostly just because trying to build them as part of > > a large bundled tree is going to be incredibly painful. > > I think this makes sense as well. Essentially, we should just say that > archful nodejs modules are "essential" (in the words of the proposal) > and therefore need to be packaged and shipped separately. > > I think a "npm2rpm" script would have to do a couple things-- does > this sound about right? > > a) recursively parse package.json to get a list of all dependencies > and tarballs, and use this to write out a list of Sources. > b) for each package, also look at package.json to see what files need > to be installed, and write out instructions for the %install section. > c) generate a "License" field for the package and also install all the > relevant licenses in %license. > > There should probably be a way to specify that a given dependency is > not being bundled too, to stop the parser from descending into that > dependency tree. I imagine that (at least during the transition to > this and maybe afterwards, in the case of archful or essential > modules) it would be very useful to have packages with a mixture of > bundled and unbundled dependencies. > > Ben Rosser > You are both right. Odd as it seems, I kept seeing nodejs-sqlite3 come up, which is archful. I had never done a real search. It turns out that the various packages I had been working on require it, and only two other packages. So ya, Ben, I like you idea. I was looking at a very skewed view. Troy ___ nodejs mailing list -- nodejs@lists.fedoraproject.org To unsubscribe send an email to nodejs-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/nodejs@lists.fedoraproject.org
Re: Proposal: Reboot nodejs library/modules in Fedora
On Tue, Jul 21, 2020 at 10:56 AM Tom Hughes via nodejs wrote: > > On 21/07/2020 15:39, Troy Dawson wrote: > > > Problem2: Many of the bundled libraries are archful, meaning they need > > to be compiled. What if an essential nodejs library was originally a > > noarch? > > Solution2: All bundled nodejs library packages will be treached as > > archful, meaning they will need to be built on each supported arch. > > I think "many" is stretching things. > > There were never more than about a dozen or so archful modules in > Fedora and most of them were not widely used. > > I suspect that archful modules are probably best kept packaged as > they are now mostly just because trying to build them as part of > a large bundled tree is going to be incredibly painful. I think this makes sense as well. Essentially, we should just say that archful nodejs modules are "essential" (in the words of the proposal) and therefore need to be packaged and shipped separately. I think a "npm2rpm" script would have to do a couple things-- does this sound about right? a) recursively parse package.json to get a list of all dependencies and tarballs, and use this to write out a list of Sources. b) for each package, also look at package.json to see what files need to be installed, and write out instructions for the %install section. c) generate a "License" field for the package and also install all the relevant licenses in %license. There should probably be a way to specify that a given dependency is not being bundled too, to stop the parser from descending into that dependency tree. I imagine that (at least during the transition to this and maybe afterwards, in the case of archful or essential modules) it would be very useful to have packages with a mixture of bundled and unbundled dependencies. Ben Rosser ___ nodejs mailing list -- nodejs@lists.fedoraproject.org To unsubscribe send an email to nodejs-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/nodejs@lists.fedoraproject.org
Re: Proposal: Reboot nodejs library/modules in Fedora
On 21/07/2020 15:39, Troy Dawson wrote: Problem2: Many of the bundled libraries are archful, meaning they need to be compiled. What if an essential nodejs library was originally a noarch? Solution2: All bundled nodejs library packages will be treached as archful, meaning they will need to be built on each supported arch. I think "many" is stretching things. There were never more than about a dozen or so archful modules in Fedora and most of them were not widely used. I suspect that archful modules are probably best kept packaged as they are now mostly just because trying to build them as part of a large bundled tree is going to be incredibly painful. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ nodejs mailing list -- nodejs@lists.fedoraproject.org To unsubscribe send an email to nodejs-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/nodejs@lists.fedoraproject.org
Proposal: Reboot nodejs library/modules in Fedora
I am sending this to the Fedora nodejs community / SIG before sending it to fedora-devel and/or the packaging committee. I'm doing that so we (the people who actually maintain, or try to maintain, the nodejs packages) have a chance to talk and discuss things without too much thread jumping. Problem1: The current way of packaging every nodejs module/library as it's own package has created library stagnation. It is extremely hard to update any of these modules due to other modules depending on already released versions. It is also very hard to add any Fedora package that requires new nodejs libraries due to the volume of package reviews that have to happen. Problem2: Due to problem1, and other reasons, some of our major nodejs packagers are orphaning their nodejs packages. This is leading to many packages being unable to install. This is starting a downward spiral of uninstallable nodejs packages. Proposed Solution: Keep nodejs and npm in Fedora. Possibly yarn too. Get rid of all non-essential nodejs libraries. Those that are essential, bundle all their build, test, and runtime nodejs libraries. Create scripts that make it easy to bundle, and maintain, these essential nodejs libraries. If needed, create scripts that make it easy for non-nodejs packages to bundle their nodejs libraries properly. LIST OF PROBLEMS TO WORK OUT: Problem1: Currently, all the libraries/modules are links under /node_modules/ and dnf doesn't like changing links into directories. Solution1: Bundled runtime libraries/modules will be in /bundled_node_modules/ and all the links in /node_modules/ will point there. Problem2: Many of the bundled libraries are archful, meaning they need to be compiled. What if an essential nodejs library was originally a noarch? Solution2: All bundled nodejs library packages will be treached as archful, meaning they will need to be built on each supported arch. Problem3: TBD Solution3: TBD Problem4: TBD Problem4: TBD What do you, the Fedora nodejs packagers and community think of this proposal? Does it sound ok? If so, let's start working through all the problems, finding solutions. If not, then let's find a different solution to our current crisis. Thanks for your time. Troy Dawson ___ nodejs mailing list -- nodejs@lists.fedoraproject.org To unsubscribe send an email to nodejs-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/nodejs@lists.fedoraproject.org
