Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
Am Fri, Jul 07, 2023 at 12:52:49AM +0530 schrieb Nilesh Patra: > > Andreas, if you remember, the code pointed out by Sébastien is > the exact same reason we had to t-p-u r-cran-shiny during freeze, See > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1035428#15 > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1035428#40 > > https://tracker.debian.org/news/1431956/accepted-r-cran-shiny-174dfsg-2deb12u1-source-into-testing-proposed-updates/ Nilesh, you somehow work like my "external memory". I just answered the issue and IMHO Sébastian is correct but this line has a long history ... Kind regards Andreas. -- http://fam-tille.de
Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
On Thu, Jul 06, 2023 at 02:24:57PM -0500, Dirk Eddelbuettel wrote: > [...] I understood some of it (not completely, possibly because of lack of background with R extensions), but thanks a lot for taking the time to explain it to me! > So if tibble does this now, it should only affect tibble itself -- unless of > course a dependent package calls directly into the native code of tibble > (possible, but rare). I found some code in vctrs which do some tibble specific stuff, like https://sources.debian.org/src/r-cran-vctrs/0.6.3-1/src/type-tibble.c/ But I don't think there are calls for native code anywhere. No idea where this comes from, then. > | Since you are building with R from unstable and tibble from testing > | (built with an older R), it chokes and works when both are new. > > Not sure I agree. That should still work. Quick check in Docker (using the > r-base image I maintain) shows it does: > > root@f39da83dba5a:/# dpkg -l r-base-core r-cran-tibble | tail -2 Very weird. That means you're unable to repro the failure in https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-rsample/35451568/log.gz Right? Would you have an idea on the github dplyr issue then? The log seems to be the same as we see in the CI https://github.com/tidyverse/dplyr/issues/6793 > It's simpy that testing has an older one and (esp in the tidyverse) things > change quickly and packages want to be in consistent cohort. But AFAICS, in both the scenarios (with and without failure) it is essentially running with the same version of tidyverse, so ideally pulling tibble from unstable and mixing it with an older tidyverse should break, right? (It's the opposite here and I'm quite confused). > | This attribute has got something to do with R extensions' entry > | points / dll compatibilty, but I simply do not have enough background > | with r-core to comment beyond this point, I'm afraid. > > Hope the above helped a little. Yes, thanks again. Best, Nilesh signature.asc Description: PGP signature
Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
On 7 July 2023 at 00:33, Nilesh Patra wrote:
| I think we are hitting this issue here:
https://github.com/tidyverse/dplyr/issues/6793
| The comment says "Looks like some package in the stack sets
R_forceSymbols(dll, TRUE)" and that package is tibble
|
| | $ grep -rnw R_forceSymbols
| | src/init.c:19: R_forceSymbols(dll, TRUE);
That mechanism should not spill from one package to the next.
What we have here is that (R) packages with compiled code can (optionally)
declare in NAMESPACE whether or not 'symbols' (ie the entrypoints for the
compiled code) should be a registered symbol (to R), or (as they used to be)
strings. That is then used in the first argument to .Call() as in eg
.Call(myfunc) as opposed to .Call("myfunc"). It has small efficiency
gains. Most packages do that now.
Now, another (less frequently-used) option is in the intialization file run
at package to also say R_forceSymbols in which case the second ("string")
form is no longer allowed. Few packages do that.
So if tibble does this now, it should only affect tibble itself -- unless of
course a dependent package calls directly into the native code of tibble
(possible, but rare).
So I sum in should not spill. I could be wrong but if there were general
spillage it would affect all R use of compiled packages and it very clearly
does not. So in short, this force symbol resolution should only affect the
symbols from the one shared (per-package) library it is set for.
| Since you are building with R from unstable and tibble from testing
| (built with an older R), it chokes and works when both are new.
Not sure I agree. That should still work. Quick check in Docker (using the
r-base image I maintain) shows it does:
root@f39da83dba5a:/# dpkg -l r-base-core r-cran-tibble | tail -2
ii r-base-core4.3.1-2 amd64GNU R core of statistical
computation and graphics system
ii r-cran-tibble 3.1.8+dfsg-1 amd64GNU R Simple Data Frames
root@f39da83dba5a:/# Rscript -e 'library(tibble); print(as_tibble(mtcars))'
# A tibble: 32 × 11
mpg cyl disphp dratwt qsecvsam gear carb
1 21 6 160110 3.9 2.62 16.5 0 1 4 4
2 21 6 160110 3.9 2.88 17.0 0 1 4 4
3 22.8 4 108 93 3.85 2.32 18.6 1 1 4 1
4 21.4 6 258110 3.08 3.22 19.4 1 0 3 1
5 18.7 8 360175 3.15 3.44 17.0 0 0 3 2
6 18.1 6 225105 2.76 3.46 20.2 1 0 3 1
7 14.3 8 360245 3.21 3.57 15.8 0 0 3 4
8 24.4 4 147.62 3.69 3.19 20 1 0 4 2
9 22.8 4 141.95 3.92 3.15 22.9 1 0 4 2
10 19.2 6 168. 123 3.92 3.44 18.3 1 0 4 4
# … with 22 more rows
root@f39da83dba5a:/#
It's simpy that testing has an older one and (esp in the tidyverse) things
change quickly and packages want to be in consistent cohort.
| This attribute has got something to do with R extensions' entry
| points / dll compatibilty, but I simply do not have enough background
| with r-core to comment beyond this point, I'm afraid.
Hope the above helped a little.
Dirk
--
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
On Thu, Jul 06, 2023 at 09:13:46PM +0200, Sébastien Villemot wrote: > > I'm not sure so please explain in more detail. dh-r was designed to put > > the lowest restriction regarding the versions. I remember some > > discussion some time ago that Dirk thought we should put stronger > > restrictions (and he is sometimes adding version restrictions manually > > that are not helpful for backporting). If I will be sure I understand > > your point exactly I can check the code and the relevant discussion. > > (Feel free to file a bug report about this and we can discuss it there > > if you think this makes more sense.) > > It comes from this line: > https://salsa.debian.org/r-pkg-team/dh-r/-/blob/master/dh/R.pm#L272 > > More precisely the “r-base-core (>= $rbase_version)” part, which > imposes an unnecessarily tight restriction on the r-base-core version. Andreas, if you remember, the code pointed out by Sébastien is the exact same reason we had to t-p-u r-cran-shiny during freeze, See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1035428#15 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1035428#40 https://tracker.debian.org/news/1431956/accepted-r-cran-shiny-174dfsg-2deb12u1-source-into-testing-proposed-updates/ Best, Nilesh signature.asc Description: PGP signature
Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
Le jeudi 06 juillet 2023 à 21:02 +0200, Andreas Tille a écrit : > Am Thu, Jul 06, 2023 at 07:08:04PM +0200 schrieb Paul Gevers: > > PS: in a private discussion I had today, we noticed that r-* packages often > > (always?) have a dependency on r-base-core with a lower limited version > > equal to the r-base-core that was used during the build. With the > > appropriate API in Provides of r-base-core, this should no longer be > > necessary and ease migrations in the future. > > Could you please give some example to make sure I understand correctly? > > > We should probably file a bug > > against dh-r (I guess) to fix that dependency. Or did we conclude that > > wrong? > > I'm not sure so please explain in more detail. dh-r was designed to put > the lowest restriction regarding the versions. I remember some > discussion some time ago that Dirk thought we should put stronger > restrictions (and he is sometimes adding version restrictions manually > that are not helpful for backporting). If I will be sure I understand > your point exactly I can check the code and the relevant discussion. > (Feel free to file a bug report about this and we can discuss it there > if you think this makes more sense.) It comes from this line: https://salsa.debian.org/r-pkg-team/dh-r/-/blob/master/dh/R.pm#L272 More precisely the “r-base-core (>= $rbase_version)” part, which imposes an unnecessarily tight restriction on the r-base-core version. -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name ⠈⠳⣄ https://www.debian.org signature.asc Description: This is a digitally signed message part
Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
Hi Paul, On Thu, Jul 06, 2023 at 08:28:45PM +0200, Paul Gevers wrote: > On 06-07-2023 19:08, Paul Gevers wrote: > > Yes, we'll take care of that where it looks sane to do so (examples of > > that are r-cran-epi); I'll manually schedule the "combined" tests, such > > that they disappear from the excuses if they then pass. > > I'm seeing in several tests where things seem to work when r-cran-tibble > from unstable is involved and fail if the version from unstable is used; > e.g. here: https://ci.debian.net/packages/r/r-cran-rsample/testing/amd64/ > > Is there something special about r-cran-tibble? It didn't grow a dependency > on r-graphics-engine according to > https://buildd.debian.org/status/fetch.php?pkg=r-cran-tibble=amd64=3.2.1%2Bdfsg-2=1688629916=0 > so it doesn't seem to be involved in that part of the discussion. I think we are hitting this issue here: https://github.com/tidyverse/dplyr/issues/6793 The comment says "Looks like some package in the stack sets R_forceSymbols(dll, TRUE)" and that package is tibble | $ grep -rnw R_forceSymbols | src/init.c:19: R_forceSymbols(dll, TRUE); Since you are building with R from unstable and tibble from testing (built with an older R), it chokes and works when both are new. This attribute has got something to do with R extensions' entry points / dll compatibilty, but I simply do not have enough background with r-core to comment beyond this point, I'm afraid. Best, Nilesh signature.asc Description: PGP signature
Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
Hi Paul, Am Thu, Jul 06, 2023 at 07:08:04PM +0200 schrieb Paul Gevers: > Yes, we'll take care of that where it looks sane to do so (examples of that > are r-cran-epi); I'll manually schedule the "combined" tests, such that they > disappear from the excuses if they then pass. OK. Thanks a lot for this service. Do you think the remaining serious tests are blockers? Should these be closed manually? > > Alternatively it would probably OK to remove all > > RC buggy r-bioc-* packages from testing since we need to upload new > > versions of these packages anyway in the pending BioC transition (I'll > > file an according bug report soon). > > If you're OK to temporarily remove r-bioc-*, than I think it's the fastest > and easiest way out, albeit not the prettiest. For sure it is not pretty, but since once week I'm fighting to non-pretty things and I'm really happy about fast and easy solutions. ;-) I have just filed a transition bug for r-bioc-* where we can discuss issues belonging to this transition. > Please all involved, hold any uploads in the r-* world until we get r-base > migrated unless it's needed (and we acked it). OK, I'll refrain from any upload of r-* packages (I'll answer your other mail about r-cran-tibble separately). > Paul > > PS: in a private discussion I had today, we noticed that r-* packages often > (always?) have a dependency on r-base-core with a lower limited version > equal to the r-base-core that was used during the build. With the > appropriate API in Provides of r-base-core, this should no longer be > necessary and ease migrations in the future. Could you please give some example to make sure I understand correctly? > We should probably file a bug > against dh-r (I guess) to fix that dependency. Or did we conclude that > wrong? I'm not sure so please explain in more detail. dh-r was designed to put the lowest restriction regarding the versions. I remember some discussion some time ago that Dirk thought we should put stronger restrictions (and he is sometimes adding version restrictions manually that are not helpful for backporting). If I will be sure I understand your point exactly I can check the code and the relevant discussion. (Feel free to file a bug report about this and we can discuss it there if you think this makes more sense.) Kind regards Andreas. -- http://fam-tille.de
Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
Hi, On 06-07-2023 19:08, Paul Gevers wrote: Yes, we'll take care of that where it looks sane to do so (examples of that are r-cran-epi); I'll manually schedule the "combined" tests, such that they disappear from the excuses if they then pass. I'm seeing in several tests where things seem to work when r-cran-tibble from unstable is involved and fail if the version from unstable is used; e.g. here: https://ci.debian.net/packages/r/r-cran-rsample/testing/amd64/ Is there something special about r-cran-tibble? It didn't grow a dependency on r-graphics-engine according to https://buildd.debian.org/status/fetch.php?pkg=r-cran-tibble=amd64=3.2.1%2Bdfsg-2=1688629916=0 so it doesn't seem to be involved in that part of the discussion. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
Hi Andreas, On 06-07-2023 16:44, Andreas Tille wrote: Am Thu, Jul 06, 2023 at 09:56:31AM +0200 schrieb Andreas Tille: Is there any chance to fix this via hinting by release team or should I rather add some "artificial" versioned Depends just to enable the whole set of r-cran-* packages to testing. Yes, we'll take care of that where it looks sane to do so (examples of that are r-cran-epi); I'll manually schedule the "combined" tests, such that they disappear from the excuses if they then pass. Alternatively it would probably OK to remove all RC buggy r-bioc-* packages from testing since we need to upload new versions of these packages anyway in the pending BioC transition (I'll file an according bug report soon). If you're OK to temporarily remove r-bioc-*, than I think it's the fastest and easiest way out, albeit not the prettiest. Please all involved, hold any uploads in the r-* world until we get r-base migrated unless it's needed (and we acked it). Paul PS: in a private discussion I had today, we noticed that r-* packages often (always?) have a dependency on r-base-core with a lower limited version equal to the r-base-core that was used during the build. With the appropriate API in Provides of r-base-core, this should no longer be necessary and ease migrations in the future. We should probably file a bug against dh-r (I guess) to fix that dependency. Or did we conclude that wrong? OpenPGP_signature Description: OpenPGP digital signature
Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing (Was: Bug#1040001: transition: r-base)
Hi again, Am Thu, Jul 06, 2023 at 09:56:31AM +0200 schrieb Andreas Tille: > > I've now re-uploaded all 6 packages that are known to be affected by the > graphics ABI change (and verified that it is set properly). I'll > continue now closing the open bugs. After rebuilding quite some packages with autopkgtest failure with r-base (4.3.1-1) it came to my mind (probably to late) that since we do not have any real transition those uploads will probably not help very much since these are not generated with dependencies that are conflicting with the versions in testing and thus the autopkgtests running against packages in testing will keep on failing. Is there any chance to fix this via hinting by release team or should I rather add some "artificial" versioned Depends just to enable the whole set of r-cran-* packages to testing. I'm also wondering whether I should upload the old r-bioc-* packages to finally get the full R stack fit for testing before I'll start the next BioC transition. Alternatively it would probably OK to remove all RC buggy r-bioc-* packages from testing since we need to upload new versions of these packages anyway in the pending BioC transition (I'll file an according bug report soon). Kind regards Andreas. -- http://fam-tille.de
Bug#1040001: transition: r-base
Hi, Am Wed, Jul 05, 2023 at 04:07:12PM + schrieb Graham Inggs: > I don't think it's possible to set up a tracker for this first > "transition", as no packages currently depend on r-graphics-engine-*. Right, this makes perfectly sense. > Please wait until r-base and dh-r are built and in the installed state > on all architectures. I've now re-uploaded all 6 packages that are known to be affected by the graphics ABI change (and verified that it is set properly). I'll continue now closing the open bugs. Kind regards Andreas. PS: @Dirk I do not mind much whether I'm mentioned in changelogs for patches[1] but if you might happen to blame me about doing things wrong again please keep in mind that I sometimes do things right. Thank you. [1] https://metadata.ftp-master.debian.org/changelogs//main/r/r-base/r-base_4.3.1-2_changelog -- http://fam-tille.de
Bug#1040001: transition: r-base
Hi Andreas On Wed, 5 Jul 2023 at 15:51, Andreas Tille wrote: > the just uploaded r-base 4.3.1-2 implements r-graphics-engine-* which is > respected by dh-r 20230705 (also just uploaded). It would be great if > you could setup transition tracker. I don't think it's possible to set up a tracker for this first "transition", as no packages currently depend on r-graphics-engine-*. > Am I understanding things correctly that I can now start uploading those > r-cran-* packages featuring bugs >autopkgtest failure with r-base (4.3.1-1) > even if the tracker is not yet setup? Please wait until r-base and dh-r are built and in the installed state on all architectures. Regards Graham
Bug#1040001: transition: r-base
Hi, the just uploaded r-base 4.3.1-2 implements r-graphics-engine-* which is respected by dh-r 20230705 (also just uploaded). It would be great if you could setup transition tracker. Am I understanding things correctly that I can now start uploading those r-cran-* packages featuring bugs autopkgtest failure with r-base (4.3.1-1) even if the tracker is not yet setup? Kind regards Andreas. -- http://fam-tille.de
Bug#1040001: transition: r-base
Paul, Graham, r-base 4.3.1-2 is now on its way. You will have to update / tweak the ben file as there is no 'r-api-4.3' tag as there is no such thing API change upstream in R itself. Filing the bug reports against the handful of packages testing the graphics engine version was the right thing to do, and their simple rebuilds now shows https://qa.debian.org/excuses.php?package=r-base has no remaining graphics bugs. This demonstrates clearly that we do not "need" a graphics api. But as I get nowhere making this argument I relented and now provide r-graphics-engine-* from r-base-core. Packages have not used it yet, of course, so the transition cannot depend on it. This of cannot resolve the remaining problems at the excuses page. It would be nice if the maintainers of the packages experiencing the breakage were looking into them. I suspect relatively simple inter-package issues. The newest BioConductor release does as always use the most recent and preceding R release, so we probably need consistent builds of BioConductor release 3.16 using the R 4.3.* version that came out just before it. And likely similar with geo-spatial stack which of then refers consistent builds of its underlying library (gdal, geos, ...) Cheers, Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Bug#1040001: transition: r-base
Hi Paul, Am Sun, Jul 02, 2023 at 10:54:03AM +0800 schrieb Paul Wise: > $ objdump -T debian/*/usr/lib/R/site-library/*/libs/*.so | grep > R_GE_checkVersionOrDie > DF *UND* Base > R_GE_checkVersionOrDie Thanks for the hint. Its implemented as suggested as per https://salsa.debian.org/r-pkg-team/dh-r/-/commit/7ad36007e575a4ec28c673ad1eae57e93ad5189e I would love to merge this with master and release a new dh-r once r-base exports the graphics ABI. Kind regards Andreas. -- http://fam-tille.de
Bug#1040001: transition: r-base
Hi Graham, Am Mon, Jul 03, 2023 at 05:24:19PM + schrieb Graham Inggs: > On Sun, 2 Jul 2023 at 19:57, Andreas Tille wrote: > > 45 > > > > serious bugs that are all caused by the non-transition while we should > > have done one. That's pretty annoying for the people who need to do the > > work (in this case basically me). > > IMHO, those autopkgtests regression bugs are useful. At least > #1039651 (in r-cran-gnm) and #1039653 (in r-cran-irkernel) appear > unrelated to the R_GE_version issue. Thank you for your uploads with fixes. > In addition, r-cran-bookdown [1] appears to break r-cran-flextable's > autopkgtests and r-cran-checkmate [2] fails its own autopkgtests on > armel, armhf and i386. These do not have bugs filed yet. I can imagine that there might be further bugs that are not filed yet. But that's not my point. I think if we would do a proper transition we can solve those issues caused by R_GE_version in one rush. All remaining things can be done manually if needed. I really wish we could make any progress in this direction. Kind regards Andreas. > [1] https://tracker.debian.org/pkg/r-cran-bookdown > [2] https://tracker.debian.org/pkg/r-cran-checkmate -- http://fam-tille.de
Bug#1040001: transition: r-base
Hi Andreas On Sun, 2 Jul 2023 at 19:57, Andreas Tille wrote: > 45 > > serious bugs that are all caused by the non-transition while we should > have done one. That's pretty annoying for the people who need to do the > work (in this case basically me). IMHO, those autopkgtests regression bugs are useful. At least #1039651 (in r-cran-gnm) and #1039653 (in r-cran-irkernel) appear unrelated to the R_GE_version issue. In addition, r-cran-bookdown [1] appears to break r-cran-flextable's autopkgtests and r-cran-checkmate [2] fails its own autopkgtests on armel, armhf and i386. These do not have bugs filed yet. Regards Graham [1] https://tracker.debian.org/pkg/r-cran-bookdown [2] https://tracker.debian.org/pkg/r-cran-checkmate
Bug#1040001: transition: r-base
Hi Paul,
Am Sat, Jul 01, 2023 at 05:21:16PM +0200 schrieb Paul Gevers:
> > So per upstream ("R Core" for short), this is clearly on the package
> > side. There is no ABI/API incompatibility: R offers graphics functions new
> > functionality, to use it one needs a rebuild _if a package decides to stop
> > and die on mismatch_.
I'd like to point out that "one needs a rebuild" is something else if
one installs a CRAN package manually or if we upload a Debian package.
A Debian package has to "survive" a set of tests (and here we
continuously disagree but right now in this actual case it has proven to
be necessary) and pass a migration procedure in a dependency tree of
packages which is just more complex than "just rebuild".
> > I so filed three bug reports last weekend against three such packages
... and I told you that this list of three packages was incomplete. By
chance I had rebuild two other packages that needed the rebuild and some
autopkgtest I was running uncovered that vdiffr simply slipped through
your attention. That's perfectly human - and since its human I'm in
favour of a technical solution which avoids such manual digging inside
the package pool.
Besides this such technical things to my experience these issues always
go with some well known pattern of "social friction" which is
demotivating for all sides. I'd like to reduce these friction points
to a minimum if there is some technical solution.
> > requesting a simple rebuild as that is in fact all it takes. (And
> > missed one that was added.) These were quickly rebuilt.
>
> While this may be true, in Debian we require that such packages express this
> relation. I understand that that's what we achieve with the proposal of
> Andreas. "Just rebuilding" is often the wrong solution (in Debian) if it
> doesn't express the relation properly.
Fully ACK.
> > So let me know what you think. If the release team thinks we must rebuild
> > across 1100 r-* packages (of which likely 400-500 are Architecture: any)
> > then I will of course work with you.
>
> I recognize that at this moment we might not need it to straighten things
> out, because of all the new version uploads, but I believe it's the right
> solution for the future, as this seems to be a recurring topic.
IMHO its not a solved case. The R pkg team is creating a list of packages
that are not up to date and a list of bugs which you can see in the second
table of [1]. This page lists
$ grep serious outdated_r-packages.txt | grep -c 4.3.1
45
serious bugs that are all caused by the non-transition while we should
have done one. That's pretty annoying for the people who need to do the
work (in this case basically me).
I would really welcome if we do it the right way even now, specifically
since there is a patch for bug #1040038 that implements a solution that
should help for now and in future.
Kind regards
Andreas.
[1]
https://salsa.debian.org/r-pkg-team/maintenance-utilities/-/blob/master/outdated_r-packages.txt
--
http://fam-tille.de
Bug#1040001: transition: r-base
On Sat, 2023-07-01 at 14:29 +0200, Andreas Tille wrote: > I've filed Bug#1040038 with the patch for r-graphics-api and updated > branch r-api-graphic branch of dh-r[1] to match the suggested patch > immediately once uploaded. I've sent a suggestion to bug #1040038 for not hard-coding the API number and extracting it from the header instead. Dirk also posted an R-native version of what I suggested to the bug too. The dh-r r-api-graphic branch also hard-codes the list of packages that should get the graphics API dependency but that list will need manual maintenance as packages add/drop use of R_GE_checkVersionOrDie. The objdump tool can export the list of dynamically imported symbols, that can be searched for use of R_GE_checkVersionOrDie and then if it is, assume that it is called with R_GE_version and add the dependency. $ objdump -T debian/*/usr/lib/R/site-library/*/libs/*.so | grep R_GE_checkVersionOrDie DF *UND* Base R_GE_checkVersionOrDie -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Bug#1040001: transition: r-base
Hi Dirk,
On 01-07-2023 16:47, Dirk Eddelbuettel wrote:
So per upstream ("R Core" for short), this is clearly on the package
side. There is no ABI/API incompatibility: R offers graphics functions new
functionality, to use it one needs a rebuild _if a package decides to stop
and die on mismatch_.
I so filed three bug reports last weekend against three such packages
requesting a simple rebuild as that is in fact all it takes. (And
missed one that was added.) These were quickly rebuilt.
While this may be true, in Debian we require that such packages express
this relation. I understand that that's what we achieve with the
proposal of Andreas. "Just rebuilding" is often the wrong solution (in
Debian) if it doesn't express the relation properly.
So let me know what you think. If the release team thinks we must rebuild
across 1100 r-* packages (of which likely 400-500 are Architecture: any)
then I will of course work with you.
I recognize that at this moment we might not need it to straighten
things out, because of all the new version uploads, but I believe it's
the right solution for the future, as this seems to be a recurring topic.
Paul
OpenPGP_signature
Description: OpenPGP digital signature
Bug#1040001: transition: r-base
This is not a bug in r-base, and does not warrant a transition.
I have written at some length about it, and (if I find some time) will expand
on it in blog post. I will also try to coordinate with upstream.
In short, R header GraphicsEngine.h [1] defines an integer constant declaring
the current versions capabilities. Graphics device packages can then opt
into what they support using conditional code. Or they can opt into checking
and aborting on any mismatch via R_GE_checkVersionOrDie (for
'R_GraphicsEngine_*)
void R_GE_checkVersionOrDie(int version)
{
if (version != R_GE_version)
error(_("Graphics API version mismatch"));
}
Which some packages do. Note that the packages _load_ and _function_ (in a
limited sense). They "simply" do not act as a graphics device, and alert the
user to rebuild the package.
So per upstream ("R Core" for short), this is clearly on the package
side. There is no ABI/API incompatibility: R offers graphics functions new
functionality, to use it one needs a rebuild _if a package decides to stop
and die on mismatch_.
I so filed three bug reports last weekend against three such packages
requesting a simple rebuild as that is in fact all it takes. (And
missed one that was added.) These were quickly rebuilt.
And as a consequence the 'excuses' list for r-base is shrinking now just as
one could expect [3]. In particular a number of packages from the ggplot2
complex are now all green ... with the exception of s390x where eg ragg (used
often in unit tests) is awaiting a rebuild [4]. Once s390x rebuilds those
packages will turn green leading into to their dependents turning green/
So in short, in my view as maintainer, a transition is overkill and a waste
of resources, cpu cycles and otherwise. I am rather confident it will
straighten itself out. (There may still be other issues with the
BioConductor release transition but I also expect those to be package-caused
and solvable.)
So let me know what you think. If the release team thinks we must rebuild
across 1100 r-* packages (of which likely 400-500 are Architecture: any)
then I will of course work with you.
But I maintain that it is not caused by the package, and not needed, as it is
a set of client-package-local differences these packages opt-in upstream and
which are solvable with a simple rebuild of those packages. Now, I
understand that _for simplicity_ we want to use this very blunt tool. But I
am old enough to remember that we also value engineering excellence. As
there is no need for brute force, I advocate against it. No more, no less.
Cheers, Dirk
[1] For us at /usr/share/R/include/R_ext/GraphicsEngine.h
[2] In the sources at src/main/engine.c
[3] https://qa.debian.org/excuses.php?package=r-base
[4] https://buildd.debian.org/status/package.php?p=r-cran-ragg
--
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Bug#1040001: transition: r-base
Hi again, just a status update: Am Sat, Jul 01, 2023 at 01:45:11PM +0200 schrieb Andreas Tille: > > I think my piece is ready. We just need to decide about a proper name > of the virtual package. I'll inject this into my proof of concept > change of dh-r. Than Dirk needs to upload another r-base package > containing the r-graphics-api-VERSION. This should not be a hard thing > to do - Dirk just stayed silent about this change since we are > discussing it. I've filed Bug#1040038 with the patch for r-graphics-api and updated branch r-api-graphic branch of dh-r[1] to match the suggested patch immediately once uploaded. Kind regards Andreas. [1] https://salsa.debian.org/r-pkg-team/dh-r/-/tree/r-api-graphic?ref_type=heads -- http://fam-tille.de
Bug#1040001: transition: r-base
Hi Paul, Am Sat, Jul 01, 2023 at 07:48:16AM +0200 schrieb Paul Gevers: > Anytime is good to ask for a transition, particularly when the transition is > already ongoing. :-) > I don't think it should surprise anyone that we prefer it to be done right. > Our preference is for option 1. Thanks for confirming. > However, if you can't get the pieces for > that option in place in a reasonable time (say, a week or two, take some > time to try), I think my piece is ready. We just need to decide about a proper name of the virtual package. I'll inject this into my proof of concept change of dh-r. Than Dirk needs to upload another r-base package containing the r-graphics-api-VERSION. This should not be a hard thing to do - Dirk just stayed silent about this change since we are discussing it. > then we prefer to get *this* transition out of the way by > means of option 2. I personally think that we are in a good situation in the beginning of the release cycle to do things right, which means option 1. But it depends from the r-base maintainer to cooperate here. > I don't think it's in anybodies interest to waste time on > option 3. ACK. I told Bas so who had spent quite some time to file bugs against lots of r-cran-* packages which are all a consequence of the not-yet-transition. > > Sorry that this transition bug is that complex. I would have loved if > > it would went more coordinated but unfortunately that's not in my hands > > and I simply try to reassemble the pieces. > > Thanks for communicating with us, much appreciated. Its always a pleasure to communicate with you. ;-) > I'll try to set a placeholder transition tracker up soon; for now, by lack > of something better, will reflect option 2. We can update that once we have > the pieces for option 1. Thanks a lot Andreas. -- http://fam-tille.de
Bug#1040001: transition: r-base
Hi Andreas, On 30-06-2023 21:35, Andreas Tille wrote: I'm not sure that we are in the right status to ask for a transition bug Anytime is good to ask for a transition, particularly when the transition is already ongoing. https://lists.debian.org/debian-r/2023/06/msg00025.html In the end of this mail three options are listed which I simply repeat here for your comfort: 1. implement the r-graphics-api-* This might be a bit complex since for the moment I do not know any means how to detect the packages that need this dependency (and how we can implement this into dh-update-R) So this might become technically complex in the first case 2. Just do a full r-api transition This would work but affects more packages than strictly necessary. My gut feeling says we will be able to finish this earlier than 1. despite technically not perfect 3. Blindly ignore the fact that we need a transition and follow the hackish workaround by using random versioned Depends as suggested by Nilesh for r-cran-epi. While I would love to hear the opinion of the release team what kind of transition (1. or 2.) should be prefered (if this is possible now at all since the affected package r-base 4.3.1 is in the archive since some time and also the most urgent packages are rebuild manually) or whether we need to fight manually through this mess (option 3.) I confirm that I agree with Johannes Ranke to prefer option 1. and do it "right" to be safe for the next time. I don't think it should surprise anyone that we prefer it to be done right. Our preference is for option 1. However, if you can't get the pieces for that option in place in a reasonable time (say, a week or two, take some time to try), then we prefer to get *this* transition out of the way by means of option 2. I don't think it's in anybodies interest to waste time on option 3. Sorry that this transition bug is that complex. I would have loved if it would went more coordinated but unfortunately that's not in my hands and I simply try to reassemble the pieces. Thanks for communicating with us, much appreciated. I'll try to set a placeholder transition tracker up soon; for now, by lack of something better, will reflect option 2. We can update that once we have the pieces for option 1. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1040001: transition: r-base
Package: release.debian.org Severity: normal User: release.debian@packages.debian.org Usertags: transition X-Debbugs-Cc: r-b...@packages.debian.org, debia...@lists.debian.org Control: affects -1 + src:r-base Hi, I'm not sure that we are in the right status to ask for a transition bug since the affected package was just uploaded some time ago by its maintainer who did not considered a proper transition. This was discussed on debia...@lists.debian.org in several postings - I try to point you to the most relevant ones https://lists.debian.org/debian-r/2023/06/msg00011.html as a response to >30 bugs against single packages all affecting the r-base migration due to (to be expected) autopkgtest errors in testing. You can basically get this list of now all RC buggy packages from the tracker page or r-base[1] https://lists.debian.org/debian-r/2023/06/msg00017.html suggests r-graphics-api-* after r-base maintainer confirmed "they cheated _a little_ and changes the graphics API" (probably meaning ABI not API) https://lists.debian.org/debian-r/2023/06/msg00016.html Reference to the docs https://lists.debian.org/debian-r/2023/06/msg00025.html In the end of this mail three options are listed which I simply repeat here for your comfort: 1. implement the r-graphics-api-* This might be a bit complex since for the moment I do not know any means how to detect the packages that need this dependency (and how we can implement this into dh-update-R) So this might become technically complex in the first case 2. Just do a full r-api transition This would work but affects more packages than strictly necessary. My gut feeling says we will be able to finish this earlier than 1. despite technically not perfect 3. Blindly ignore the fact that we need a transition and follow the hackish workaround by using random versioned Depends as suggested by Nilesh for r-cran-epi. https://lists.debian.org/debian-r/2023/06/msg00027.html Confirmation for option 1. While I would love to hear the opinion of the release team what kind of transition (1. or 2.) should be prefered (if this is possible now at all since the affected package r-base 4.3.1 is in the archive since some time and also the most urgent packages are rebuild manually) or whether we need to fight manually through this mess (option 3.) I confirm that I agree with Johannes Ranke to prefer option 1. and do it "right" to be safe for the next time. To support this idea I just commited some proof of concept change to dh-r which would support injecting a virtual package in case r-base would define it. This requires confirmation of the r-base maintainer. Sorry that this transition bug is that complex. I would have loved if it would went more coordinated but unfortunately that's not in my hands and I simply try to reassemble the pieces. Kind regards Andreas. [1] https://tracker.debian.org/pkg/r-base [2] https://salsa.debian.org/r-pkg-team/dh-r/-/commit/f79e2573a59c1ff01c526a7dcf15b7f85263cc29 Ben file: title = "r-base"; is_affected = ; is_good = ; is_bad = ;
