Bug#1064824: node-d3: fails to export map and other functions
On Tue, Mar 05, 2024 at 05:39:00PM +0530, Nilesh Patra wrote: > On Mon, Mar 04, 2024 at 09:18:01PM +, Julian Gilbey wrote: > [...] > > (!) Conflicting re-exports > > "index.js" re-exports "map" from both > > "../../../usr/share/nodejs/d3-array/src/index.js" and > > "../../../usr/share/nodejs/d3-collection/src/index.js" (will be ignored). > > created dist/d3.min.js in 4.2s > > - > > I have pushed a commit to salsa that hopefully fixes this - can you please > try with the same and see if that > helps you somewhat? That's perfect, thanks! And such a simple solution :-) (Though I don't pretend to understand how the patch works!) The taskflow webserver now works exactly as intended (as far as I can tell). > > So it's specifically "map" that is problematic, and I just happen to > > have stumbled upon it: d3 v5 depends on d3-array version 1, but the > > version of node-d3-array in unstable is 3.2.0+~cs5.0.6-2, and this is > > causing the conflict. > > > > I don't know the best way to fix this. node-d3-array version was > > upgraded from 1.2.4 to 3.x about two years ago, so d3 would have had > > this bug since then, but I'm the first one to stumble upon it :-/ > > > > Perhaps we could package node-d3-array-1 (version 1.2.4) and have > > node-d3 build-depends on that? > > I tried to embed it and realised it is creating an unholy mess. I got it > working eventually > but it can open a can of worms sometime later. Maybe packaging the older > version > would be the way to go if my fix above does not work. > > I've added yadd to the thread for more qualified advice. It might be a good idea to do this anyway, but given that no-one else has noticed a problem in > 2 years suggests it's not worth the effort. Best wishes, Julian
Bug#1064824: node-d3: fails to export map and other functions
On Mon, Mar 04, 2024 at 09:18:01PM +, Julian Gilbey wrote: > index.js → dist/d3.js... > (!) Conflicting re-exports > "index.js" re-exports "map" from both > "../../../usr/share/nodejs/d3-array/src/index.js" and > "../../../usr/share/nodejs/d3-collection/src/index.js" (will be ignored). > created dist/d3.js in 1.9s > > index.js → dist/d3.min.js... > (!) Conflicting re-exports > "index.js" re-exports "map" from both > "../../../usr/share/nodejs/d3-array/src/index.js" and > "../../../usr/share/nodejs/d3-collection/src/index.js" (will be ignored). > created dist/d3.min.js in 4.2s > - I have pushed a commit to salsa that hopefully fixes this - can you please try with the same and see if that helps you somewhat? > So it's specifically "map" that is problematic, and I just happen to > have stumbled upon it: d3 v5 depends on d3-array version 1, but the > version of node-d3-array in unstable is 3.2.0+~cs5.0.6-2, and this is > causing the conflict. > > I don't know the best way to fix this. node-d3-array version was > upgraded from 1.2.4 to 3.x about two years ago, so d3 would have had > this bug since then, but I'm the first one to stumble upon it :-/ > > Perhaps we could package node-d3-array-1 (version 1.2.4) and have > node-d3 build-depends on that? I tried to embed it and realised it is creating an unholy mess. I got it working eventually but it can open a can of worms sometime later. Maybe packaging the older version would be the way to go if my fix above does not work. I've added yadd to the thread for more qualified advice. Best, Nilesh signature.asc Description: PGP signature
Bug#1064824: node-d3: fails to export map and other functions
On Mon, Mar 04, 2024 at 09:18:01PM +, Julian Gilbey wrote: > [...] > So it's specifically "map" that is problematic, and I just happen to > have stumbled upon it: d3 v5 depends on d3-array version 1, but the > version of node-d3-array in unstable is 3.2.0+~cs5.0.6-2, and this is > causing the conflict. > > I don't know the best way to fix this. node-d3-array version was > upgraded from 1.2.4 to 3.x about two years ago, so d3 would have had > this bug since then, but I'm the first one to stumble upon it :-/ > > Perhaps we could package node-d3-array-1 (version 1.2.4) and have > node-d3 build-depends on that? I was just wondering if there's a simpler way than having a new package: have d3-array 1.2.4 being an unexported component of the node-d3 source package, and instead of Build-Depends: node-d3-array, have a Build-Conflicts: node-d3-array. But then I discovered that the binary package node-d3 depends on node-d3-array, as do 7 other packages in testing (I haven't checked unstable). I'm guessing that most of those dependencies probably need version 1.x of d3-array, so it may be that having a node-d3-array-1 (or -v1) package is actually the way forward in this situation. Best wishes, Julian
Bug#1064824: node-d3: fails to export map and other functions
Hi Nilesh, On Tue, Mar 05, 2024 at 02:06:08AM +0530, Nilesh Patra wrote: > > This gives lots of differences still; stripping down to just the > > differences still has many, many differences: some new exports not in > > the original d3, and some lost exports; the list begins: > > $ diff -u /tmp/d3-npm.exports.trimmed /tmp/d3-debian.exports.trimmed > > > > +exports.Adder = Adder; > > -exports.bisect = bisectRight; > > +exports.bin = bin; > > +exports.bisect = bisect; > > +exports.bisectCenter = bisectCenter; > > +exports.blur2 = blur2; > > +exports.blur = blur; > > +exports.blurImage = blurImage; > > +exports.count = count; > > -exports.csvFormatRow = csvFormatRow; > > -exports.csvFormatValue = csvFormatValue; > > $ cat /tmp/d3-debian.exports.trimmed | egrep --color > '(bisectRight|csvFormatRow|csvFormatValue)' > exports.bisectRight = bisectRight; > exports.csvFormatRows = csvFormatRows; > > Some of the diff entries are false positive -- it is just that entries are > not identical across these > files and despite sorting them, you do not get the exact picture of the diff > in exports. Well spotted, thanks! I'll snip the discussion of csvFormatValue... > [...] > Which is why. Seems the versions of dev dependencies have not been > appropriately tightened by the upstream > so we are running into weird surprises like these. Re-compiling node-d3 again > now should fixup this export however. Great! > These minor deltas in exports are more or less due to > version differences in different d3 plugins. > > ... > > Background to this: I'm trying to package a new package which provides > > a web server to visualise some data. The package includes a few > > precompiled JavaScript libraries obtained from npmjs.com, and the > > server works fine with them. But following Debian policy, I need to > > replace those with the source packages. And the server then doesn't > > work. > > > > The JavaScript libraries which the package uses are: d3 v5.16.0, > > d3-tip, apparently v0.9.1, along with jQuery and bootstrap4. I have > > replaced all of these with the versions in the corresponding Debian > > packages (and I've just uploaded a new version of d3-tip, thinking > > that that was the cause of the bug). > > > > When visiting the served web page, the console log gives the error > > message: > > > > Uncaught (in promise) TypeError: t.map is not a function > >n http://localhost:8080/js/d3/d3-tip.min.js:1 > >main http://localhost:8080/js/index.js:848 > >async* http://localhost:8080/js/index.js:993 > > > > (This has changed from the original bug report as the minimised new > > version of d3-tip has t.map instead of h.map.) > > > > d3-tip.js requires d3-collection, from which it calls a map function. > > I tried replacing d3-tip.min.js with the pre-packaged version rather > > than the (newly built) Debian version, but that did not help. I > > reverted that change and instead replaced d3.v5.min.js (which is a > > copy of /usr/share/nodejs/d3/dist/d3.min.js) with the version provided > > by upstream, which is a verbatim copy of the npmjs.com d3.min.js, and > > the server then worked perfectly. So this told me that it is the > > Debian compiled d3 which is not working correctly. > > Julian, I am very confused by that wording - from what I could gauge, your > target package does not work with debian libs but it does work with npmjs, > yes? I'm sorry I wasn't clear. Yes, that's it: I'm trying to package taskflow for Debian (it's a dependency of something else that I'm trying to package), and it provides a profiler visualisation tool in the form of a webserver. The upstream package (https://github.com/taskflow/taskflow) has d3.v5.min.js and d3-tip.min.js included (in tfprof/js/d3), and these are bitwise identical to the npmjs versions (versions 5.16.0 and 0.9.1). The webserver works using these versions. To satisfy Debian policy, I replaced these with the versions found in libjs-d3-tip and node-d3, but then the webserver fails to produce a usuable visualisation. My exploration, described above, was that "map" was not exported from d3-collection, and that led me to explore whether this was a unique missing export or not; I discovered that it was not. > In that case linking your target package and listing the exact steps to > that error can help someone debug it in more detail as to what might be > missing. I've just rebuilt node-d3 locally from (Debian) source on an unstable schroot, and I think I may have found the source of the bug: the build reads: - dh_auto_build --buildsystem=nodejs No build command found, searching known files No build command found, searching known files Found debian/nodejs/d3-sankey/build cd ./d3-sankey && sh -ex ../debian/nodejs/d3-sankey/build + rollup -c src/index.js → dist/d3-sankey.js... created dist/d3-sankey.js in 120ms src/index.js → dist/d3-sankey.min.js... created dist/d3-sankey.min.js in 561ms Found debian/nodejs/./build cd ./. && sh -e
Bug#1064824: node-d3: fails to export map and other functions
> This gives lots of differences still; stripping down to just the > differences still has many, many differences: some new exports not in > the original d3, and some lost exports; the list begins: > $ diff -u /tmp/d3-npm.exports.trimmed /tmp/d3-debian.exports.trimmed > > +exports.Adder = Adder; > -exports.bisect = bisectRight; > +exports.bin = bin; > +exports.bisect = bisect; > +exports.bisectCenter = bisectCenter; > +exports.blur2 = blur2; > +exports.blur = blur; > +exports.blurImage = blurImage; > +exports.count = count; > -exports.csvFormatRow = csvFormatRow; > -exports.csvFormatValue = csvFormatValue; $ cat /tmp/d3-debian.exports.trimmed | egrep --color '(bisectRight|csvFormatRow|csvFormatValue)' exports.bisectRight = bisectRight; exports.csvFormatRows = csvFormatRows; Some of the diff entries are false positive -- it is just that entries are not identical across these files and despite sorting them, you do not get the exact picture of the diff in exports. However I did not find csvFormatValue and I dug a little bit into it. Seems this is sourced from node-d3-dsv and exported with index.js: https://github.com/d3/d3/blob/v5.16.0/index.js And checking on current unstable | [/tmp/node-d3-dsv]$ git checkout debian/1.2.0+_1.2.3-1 | HEAD is now at 76229a9 releasing package node-d3-dsv version 1.2.0+~1.2.3-1 | [76229a9][/tmp/node-d3-dsv]$ grep -irn csvFormatValue | types-d3-dsv/index.d.ts:195:// csvFormatValue(...) | types-d3-dsv/index.d.ts:202:export function csvFormatValue(value: string): string; | src/csv.js:11:export var csvFormatValue = csv.formatValue; | src/index.js:2:export {csvParse, csvParseRows, csvFormat, csvFormatBody, csvFormatRows, csvFormatRow, csvFormatValue} from "./csv.js"; So this should work, weird. I wanted to dig in, as per the build log: https://buildd.debian.org/status/fetch.php?pkg=node-d3&arch=all&ver=5.16.0%2B%7Ecs5.28.10-1&stamp=1693715544&raw=0 | Selecting previously unselected package node-d3-dsv. | Preparing to unpack .../317-node-d3-dsv_1.1.1-9_all.deb ... | Unpacking node-d3-dsv (1.1.1-9) ... | Selecting previously unselected packa And | [76229a9][/tmp/node-d3-dsv]$ git checkout debian/1.1.1-9 | Previous HEAD position was 76229a9 releasing package node-d3-dsv version 1.2.0+~1.2.3-1 | HEAD is now at 84fbfce releasing package node-d3-dsv version 1.1.1-9 | [84fbfce][/tmp/node-d3-dsv]$ grep -irn csvFormatValue | wc -l | 0 Which is why. Seems the versions of dev dependencies have not been appropriately tightened by the upstream so we are running into weird surprises like these. Re-compiling node-d3 again now should fixup this export however. :-( These minor deltas in exports are more or less due to version differences in different d3 plugins. > ... > Background to this: I'm trying to package a new package which provides > a web server to visualise some data. The package includes a few > precompiled JavaScript libraries obtained from npmjs.com, and the > server works fine with them. But following Debian policy, I need to > replace those with the source packages. And the server then doesn't > work. > > The JavaScript libraries which the package uses are: d3 v5.16.0, > d3-tip, apparently v0.9.1, along with jQuery and bootstrap4. I have > replaced all of these with the versions in the corresponding Debian > packages (and I've just uploaded a new version of d3-tip, thinking > that that was the cause of the bug). > > When visiting the served web page, the console log gives the error > message: > > Uncaught (in promise) TypeError: t.map is not a function >n http://localhost:8080/js/d3/d3-tip.min.js:1 >main http://localhost:8080/js/index.js:848 >async* http://localhost:8080/js/index.js:993 > > (This has changed from the original bug report as the minimised new > version of d3-tip has t.map instead of h.map.) > > d3-tip.js requires d3-collection, from which it calls a map function. > I tried replacing d3-tip.min.js with the pre-packaged version rather > than the (newly built) Debian version, but that did not help. I > reverted that change and instead replaced d3.v5.min.js (which is a > copy of /usr/share/nodejs/d3/dist/d3.min.js) with the version provided > by upstream, which is a verbatim copy of the npmjs.com d3.min.js, and > the server then worked perfectly. So this told me that it is the > Debian compiled d3 which is not working correctly. Julian, I am very confused by that wording - from what I could gauge, your target package does not work with debian libs but it does work with npmjs, yes? In that case linking your target package and listing the exact steps to that error can help someone debug it in more detail as to what might be missing. Best, Nilesh signature.asc Description: PGP signature
Bug#1064824: node-d3: fails to export map and other functions
reassign 1064824 node-d3 5.16.0+~cs5.28.10-1 severity 1064824 serious retitle 1064824 node-d3: fails to export map and probably other functions thanks (I'm raising this to severity "serious" because it breaks a different package.) (For background to the discovery of this bug, see below.) The Debian version of d3.js does not export "map", which it should. This breaks the libjs-d3-tip package. In fact, the export lists are significantly different: $ grep '^exports' d3-npm/dist/d3.js | sort > /tmp/d3-npm.exports $ grep ^exports /usr/share/nodejs/d3/dist/d3.js | sort > /tmp/d3-debian.exports $ diff -u /tmp/d3-npm.exports /tmp/d3-debian.exports [520 lines, lots of differences are due to "$1" type suffixes/ $ sed -e 's/\$[0-9]*;/;/' /tmp/d3-npm.exports > /tmp/d3-npm.exports.trimmed $ sed -e 's/\$[0-9]*;/;/' /tmp/d3-debian.exports > /tmp/d3-debian.exports.trimmed $ diff -u /tmp/d3-npm.exports.trimmed /tmp/d3-debian.exports.trimmed This gives lots of differences still; stripping down to just the differences still has many, many differences: some new exports not in the original d3, and some lost exports; the list begins: +exports.Adder = Adder; -exports.bisect = bisectRight; +exports.bin = bin; +exports.bisect = bisect; +exports.bisectCenter = bisectCenter; +exports.blur2 = blur2; +exports.blur = blur; +exports.blurImage = blurImage; +exports.count = count; -exports.csvFormatRow = csvFormatRow; -exports.csvFormatValue = csvFormatValue; Now, one *might* guess that the reason for this is that d3 imports lots of d3-* modules, and these have been updated and changed since d3 v5.16.0 was released, so these changes are reflected in the new list of exports (though people assuming that the Debian version of d3 v5.16.0 reflects the upstream version will then be confused and possibly frustrated). But that doesn't explain the absence of map, which comes from (node-)d3-collection. For /usr/share/nodejs/d3-collection/dist/d3-collection.js reads: exports.entries = entries; exports.keys = keys; exports.map = map; exports.nest = nest; exports.set = set; exports.values = values; So I cannot fathom why map is not being exported by d3 itself. There may be other cases of this behaviour, but I have not investigated further. I am quite stumped by this. Background to this: I'm trying to package a new package which provides a web server to visualise some data. The package includes a few precompiled JavaScript libraries obtained from npmjs.com, and the server works fine with them. But following Debian policy, I need to replace those with the source packages. And the server then doesn't work. The JavaScript libraries which the package uses are: d3 v5.16.0, d3-tip, apparently v0.9.1, along with jQuery and bootstrap4. I have replaced all of these with the versions in the corresponding Debian packages (and I've just uploaded a new version of d3-tip, thinking that that was the cause of the bug). When visiting the served web page, the console log gives the error message: Uncaught (in promise) TypeError: t.map is not a function n http://localhost:8080/js/d3/d3-tip.min.js:1 main http://localhost:8080/js/index.js:848 async* http://localhost:8080/js/index.js:993 (This has changed from the original bug report as the minimised new version of d3-tip has t.map instead of h.map.) d3-tip.js requires d3-collection, from which it calls a map function. I tried replacing d3-tip.min.js with the pre-packaged version rather than the (newly built) Debian version, but that did not help. I reverted that change and instead replaced d3.v5.min.js (which is a copy of /usr/share/nodejs/d3/dist/d3.min.js) with the version provided by upstream, which is a verbatim copy of the npmjs.com d3.min.js, and the server then worked perfectly. So this told me that it is the Debian compiled d3 which is not working correctly. Best wishes, Julian