Bug#977781: [Pkg-javascript-devel] Bug#977781: Bug#977781: Bug#977781: real issue is, it does not pull not-yet-cached modules
Control: tags -1 pending On Tue, Dec 22, 2020 at 1:20 pm, Akshay S Dinesh wrote: gbp:error: upstream/1.22.10+_cs18.39.16 is not a valid treeish indeed your fork only has 33 tags, whereas js-team/node-yarnpkg has 37. Please pull those tags and push them to your fork, then force restart the CI pipeline. This would help us to see if your proposed fix works. Pushing tags indeed helped extract sources. But build fails due to incompatibilities with upstream files As to upgrading tar-fs, it is possible that some other dependency we updated here and there requires it. Upstream are currently using 1.16.3: https://github.com/yarnpkg/yarn/blob/master/yarn.lock#L7137 whereas we were already using version 2 If we want to upgrade tar-fs we need to upgrade its sources and then import them in upstream and master branches: https://wiki.debian.org/Javascript/GroupSourcesTutorial I think Praveen locked the dependency on tar-fs to version 1.x in https://salsa.debian.org/js-team/node-yarnpkg/-/commit/8f44f7d238f4330b89d3b62c2cc369bb909459a6 I'm unsure if that's because upstream yarn mentions 1.x tar-fs as dependency or because there are other dependencies to 1.x tar-fs. The API of tar-fs doesn't change between 1.x and 2.x, but the dependencies do. And the dependencies we have in debian make it such that 1.x doesn't work. I tried following GroupSourcesTutorial and gbp manual. It is becoming very confusing for me with many version numbers and tags floating around. So I'll have to leave this here for possibly Praveen to pick up. To update a component, you have to update debian/watch and run uscan, then gbp import orig --pristine-tar. - https://registry.npmjs.org/tar-fs/-/tar-fs-(1(?:.[\d\.]+)?)@ARCHIVE_EXT@ checksum + https://registry.npmjs.org/tar-fs/-/tar-fs-(2(?:.[\d\.]+)?)@ARCHIVE_EXT@ checksum But it seems ctype=nodejs is not working as expected (it is supposed to download versions matching with package.json but that does not seem to be working). I can install packages now. Uploading the fixed version.
Bug#977781: [Pkg-javascript-devel] Bug#977781: Bug#977781: real issue is, it does not pull not-yet-cached modules
gbp:error: upstream/1.22.10+_cs18.39.16 is not a valid treeish indeed your fork only has 33 tags, whereas js-team/node-yarnpkg has 37. Please pull those tags and push them to your fork, then force restart the CI pipeline. This would help us to see if your proposed fix works. Pushing tags indeed helped extract sources. But build fails due to incompatibilities with upstream files As to upgrading tar-fs, it is possible that some other dependency we updated here and there requires it. Upstream are currently using 1.16.3: https://github.com/yarnpkg/yarn/blob/master/yarn.lock#L7137 whereas we were already using version 2 If we want to upgrade tar-fs we need to upgrade its sources and then import them in upstream and master branches: https://wiki.debian.org/Javascript/GroupSourcesTutorial I think Praveen locked the dependency on tar-fs to version 1.x in https://salsa.debian.org/js-team/node-yarnpkg/-/commit/8f44f7d238f4330b89d3b62c2cc369bb909459a6 I'm unsure if that's because upstream yarn mentions 1.x tar-fs as dependency or because there are other dependencies to 1.x tar-fs. The API of tar-fs doesn't change between 1.x and 2.x, but the dependencies do. And the dependencies we have in debian make it such that 1.x doesn't work. I tried following GroupSourcesTutorial and gbp manual. It is becoming very confusing for me with many version numbers and tags floating around. So I'll have to leave this here for possibly Praveen to pick up. Akshay
Bug#977781: [Pkg-javascript-devel] Bug#977781: Bug#977781: real issue is, it does not pull not-yet-cached modules
Hi Akshay, many thanks for the debugging ! see below Il 22/12/20 06:06, Akshay S Dinesh ha scritto: There are some 4 pipes before the finish event. I'm looking through each one of them to see if there's a mismatch. It seems to be tar-fs Please see https://salsa.debian.org/js-team/node-yarnpkg/-/merge_requests/4 I've just downloaded the latest version from the github of tar-fs and replace in the directory. Not sure if this is the way to do it. It's a pity the salsa Continuous Integration has failed on your fork. I noticed this message in the extract-source job log: gbp:error: upstream/1.22.10+_cs18.39.16 is not a valid treeish indeed your fork only has 33 tags, whereas js-team/node-yarnpkg has 37. Please pull those tags and push them to your fork, then force restart the CI pipeline. This would help us to see if your proposed fix works. As to upgrading tar-fs, it is possible that some other dependency we updated here and there requires it. Upstream are currently using 1.16.3: https://github.com/yarnpkg/yarn/blob/master/yarn.lock#L7137 whereas we were already using version 2 If we want to upgrade tar-fs we need to upgrade its sources and then import them in upstream and master branches: https://wiki.debian.org/Javascript/GroupSourcesTutorial Paolo
Bug#977781: [Pkg-javascript-devel] Bug#977781: real issue is, it does not pull not-yet-cached modules
There are some 4 pipes before the finish event. I'm looking through each one of them to see if there's a mismatch. It seems to be tar-fs Please see https://salsa.debian.org/js-team/node-yarnpkg/-/merge_requests/4 I've just downloaded the latest version from the github of tar-fs and replace in the directory. Not sure if this is the way to do it.
Bug#977781: [Pkg-javascript-devel] Bug#977781: real issue is, it does not pull not-yet-cached modules
> >I think the real issue is that it does not pull not-yet-cached
modules.
>
I did some console.log debugging yesterday.
The tarball-fetcher, for whatever be the reason, doesn't ever trigger
the .on('finish' event
https://salsa.debian.org/js-team/node-yarnpkg/-/blob/master/src/fetchers/tarball-fetcher.js#L164
There are some 4 pipes before the finish event. I'm looking through each
one of them to see if there's a mismatch.
Bug#977781: [Pkg-javascript-devel] Bug#977781: real issue is, it does not pull not-yet-cached modules
On Mon, 21 Dec 2020 14:24:57 +0530 Pirate Praveen wrote: > >I think the real issue is that it does not pull not-yet-cached modules. > > I think the failure is after it downloaded. I can see some files in ~/.cache/yarn created after it is cleaned after yarnpkg add. Could it be a problem with extracting the tar ? You were correct, these are empty directories, so the failure seems to be in downloading. $ ls ~/.cache/yarn/v6/ npm-d3-color-2.0.0-8d625cab42ed9b8f601a1760a389f7ea9189d62e-integrity pravi@mahishi:/tmp$ rm -rf ~/.cache/yarn/v6/ pravi@mahishi:/tmp$ yarnpkg add d3-color yarn add v1.22.10 info No lockfile found. [1/4] Resolving packages... [2/4] Fetching packages... [-] 0/1pravi@mahishi:/tmp$ ls ~/.cache/yarn/v6/ npm-d3-color-2.0.0-8d625cab42ed9b8f601a1760a389f7ea9189d62e-integrity pravi@mahishi:/tmp$ ls ~/.cache/yarn/v6/npm-d3-color-2.0.0-8d625cab42ed9b8f601a1760a389f7ea9189d62e-integrity/ node_modules pravi@mahishi:/tmp$ ls ~/.cache/yarn/v6/npm-d3-color-2.0.0-8d625cab42ed9b8f601a1760a389f7ea9189d62e-integrity/node_modules/ d3-color pravi@mahishi:/tmp$ ls ~/.cache/yarn/v6/npm-d3-color-2.0.0-8d625cab42ed9b8f601a1760a389f7ea9189d62e-integrity/node_modules/d3-color/ pravi@mahishi:/tmp$
Bug#977781: [Pkg-javascript-devel] Bug#977781: real issue is, it does not pull not-yet-cached modules
On 2020, ഡിസംബർ 21 1:46:31 PM IST, Paolo Greppi wrote: >Hi Pirate, > >what you want to put in ~/.yarnrc.yml could be installed globally to >/etc/yarn/config or /etc/yarnrc, but that does not actually fix it. I agree, I wanted to see if it can be used to update to yarn 2 (so yarnpkg just becomes an installer for yarn 2). I was able to use yarn 2 successfully in gitlab. This commit makes the switch to yarn 2. https://salsa.debian.org/ruby-team/gitlab/-/commit/43df5d0233cef3936b3d5abab334c5fdeeafa717 >I think the real issue is that it does not pull not-yet-cached modules. I think the failure is after it downloaded. I can see some files in ~/.cache/yarn created after it is cleaned after yarnpkg add. Could it be a problem with extracting the tar ? >To reproduce: > > # clear cache > rm -rf ~/.cache/yarn > # actual test > cd `mktemp -d` > yarnpkg init -y > yarnpkg add d3-color > >Adding the nodeLinker: "node-modules" option to ~/.yarnrc.yml or the global >locations does not help. It helps if you switch to yarn 2 by running yarnpkg set version berry. See the patch linked above for gitlab. It was failing with yarn 1, but worked after switching to berry. We cannot set version to berry globally, it has to be set per project. >It would be interesting to debug the JavaScript execution after it prints >"Fetching packages..." > I think we need to revert this patch and fix the error. https://salsa.debian.org/js-team/node-yarnpkg/-/commit/c122f5c26eefb0b68e0c680eb0d65aae86ef4f9e -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Bug#977781: real issue is, it does not pull not-yet-cached modules
Hi Pirate, what you want to put in ~/.yarnrc.yml could be installed globally to /etc/yarn/config or /etc/yarnrc, but that does not actually fix it. I think the real issue is that it does not pull not-yet-cached modules. To reproduce: # clear cache rm -rf ~/.cache/yarn # actual test cd `mktemp -d` yarnpkg init -y yarnpkg add d3-color Adding the nodeLinker: "node-modules" option to ~/.yarnrc.yml or the global locations does not help. It would be interesting to debug the JavaScript execution after it prints "Fetching packages..." Paolo
