Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro...
On Thu, 17 Oct 2019, Ximin Luo wrote: > Who is using reprepro to archive Debian Rust packages? That's the first Anybody who is mirroring Debian unstable with reprepro right now. I have no special interest in rust, but I do maintain a debian derivative that we build with reprepro merging debian testing and our own packages (and we mirror unstable as well because we cherry-pick fixes from unsatble from time to time, hence the reason why I discovered this before it has his testing). > time I've heard of this. I suspect this is a small number (its popcon > [1] is less than that of rustc itself), and that they will be perfectly > happy to upgrade to a fixed version of reprepro. Such a fixed version will not magically appear and will not be magically deployed. I have no problem upgrading to a newer reprepro once there's a fixed version but I do still believe that your use of Provides is abusive and that you should rethink the approach. Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/
Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro...
Raphael Hertzog: > [..] > > It might not be as flexible as the current approach as it might require > rebuilds when the package providing the interface changes, but that's > quite usual in Debian. > This isn't suitable for Rust, there will be too many rebuilds needed (basically half the ecosystem for every change, which happens 4-5 times a month) and I and I suspect everyone else will stop maintaining Rust packages if we are forced to do this. > [..] > > But reprepro is in wide use and your new package is the first one to > trigger the limit. You can't just ignore the reality, you have to cope > with the fact that we have reprepro users and that we can't deploy > a package with 270K-long Provides header currently. > Who is using reprepro to archive Debian Rust packages? That's the first time I've heard of this. I suspect this is a small number (its popcon [1] is less than that of rustc itself), and that they will be perfectly happy to upgrade to a fixed version of reprepro. [1] https://qa.debian.org/developer.php?login=brl...@debian.org#reprepro X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git
Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro...
Ansgar: > Ximin Luo writes: >> Raphael Hertzog: >>> Don't abuse the "Provides" field when you have such a volume of >>> interfaces to document. >> >> Can you please explain why 256 KB provides field is "abuse"? > > The Packages index is a shared resource by all packages and every Debian > user has to download and process the full packages index; adding > excessive amounts of data should therefore be avoided. (The 256 KB > added to the Packages index are larger than the entire source > (compressed) source package...) > Only a few (<10) packages out of 600+ rust packages have very large Provides fields. The main Debian tools are coping with it fine. It's one derivative unofficial tool that's unable to cope. I don't see why we should introduce artificial limits (and increase workload) in order to cater to one old tool. >> Do you have some concrete suggestions on how to improve the tool to >> reduce this "abuse"? > > Don't generate virtual packages (Provides) for every feature; don't > generate four virtual packages for every feature. > Simply "not doing this" quite literally will break everything in the Debian Rust ecosystem of packages, and is quite directly analogous to asking dpkg to "ignore all Provides". Several alternatives were explored in the past and the current option was settled upon in #901827. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git
Bug#942487: [Pkg-rust-maintainers] Bug#942487: Bug#942487: Bug#942487: rust-web-sys: Provides header is more than 256K long and it breaks reprepro...
On Thu, Oct 17, 2019 at 09:40:59PM +0200, Raphael Hertzog wrote: > Look, I'm not a cargo/rust expert, I won't design the tool for you but I > implemented dpkg-gensymbols and the symbols support for dpkg-shlibdeps > and I'm pretty confident that such a solution can work for your case too. > > We are not adding a provides to libc6 for each symbol that the library is > exporting. And you should not add a provides for each "interface" (or > whatever it's called in rust) that a package is providing. > > You should export the list of interfaces in a separate metadata file thas > is not part of the generated "Packages" file and you should have a tool to > generate the binary dependencies pointing back to the correct package that > is exporting the interface. > > It might not be as flexible as the current approach as it might require > rebuilds when the package providing the interface changes, but that's > quite usual in Debian. I'm suspecting there might be some confusion here, just to make sure we're all on the same page: This isn't about symbols and we aren't generating virtual packages per interface intentionally, we're generating virtual packages based on something the rust world is referring to as "features", which are basically on/off switches that are used to drive conditional compilation and (this is important) optional dependency resolution. Not every feature gets its own package, this only happens if a feature has additional optional dependencies. For everything else we add a provides entry to keep the number of binary packages in debian low. The reason we need to depend on packages with their features specified is so we resolve the dependency tree of a rust package correctly, in a loop free way, and with support for semver similar to how cargo does it. This has worked nicely for >600 packages so far. The reason web-sys upstream has this many features and lists interfaces explicitly is likely due to LTO limitations related to wasm and the required js glue, not something we have much control over.
