Bug#994457: r-cran-gnm autopkgtest needs to be adapted for lapack 3.10
Dear Sébastien, Thanks for the patch, that will certainly do the job till I get this fixed in the next gnm release - sorry I've not got to it yet. Best wishes, Heather On Mon, Oct 11, 2021, at 5:14 PM, Sébastien Villemot wrote: > Dear Heather, > > Please find attached the patch that I am going to apply to r-cran-gnm > in Debian. > > With this patch applied, the test now works with both lapack 3.9 and > 3.10. > > Feel free to use it (or not) for the next release of gnm. > > Best wishes, > > Le lundi 27 septembre 2021 à 12:01 +0100, Heather Turner a écrit : >> Hi Andreas, >> >> Thanks for forwarding this bug report and thanks to Sébastien for the >> detailed and accurate analysis. I will need to submit an update to CRAN, >> which should be feasible in the next week or two. Is there a deadline that I >> need to work to? >> >> Best wishes, >> >> Heather >> >> On Mon, Sep 27, 2021, at 10:12 AM, Andreas Tille wrote: >> > Control: tags -1 upstream >> > Control: forwarded -1 Heather Turner >> > >> > Hi Heather, >> > >> > the Debian packaged gnm recieved a bug report about a failing test in >> > connection with the upgrade to lapack 3.10.0 on the machine running the >> > test. Please read the bug report below. >> > >> > We admit we need your help to solve this issue that might affect >> > other systems as well. >> > >> > Kind regards >> > >> > Andreas. >> > >> > On Thu, Sep 16, 2021 at 10:59:19AM +0200, Sébastien Villemot wrote: >> > > Package: r-cran-gnm >> > > Version: 1.1-1-2 >> > > Severity: serious >> > > Tags: sid bookworm >> > > X-Debbugs-CC: debian...@lists.debian.org >> > > User: debian...@lists.debian.org >> > > Usertags: needs-update >> > > >> > > Dear Maintainer, >> > > >> > > Since the upload of lapack 3.10.0-1, the autopkgtest of r-cran-gnm >> > > fails in unstable. See for example: >> > > https://ci.debian.net/data/autopkgtest/unstable/amd64/r/r-cran-gnm/15155026/log.gz >> > > >> > > More precisely, test-biplot.R fails, because some results have the >> > > opposite sign compared to the one which is expected. >> > > >> > > My understanding is that this comes from the SVD of barleyMatrix in >> > > that test file, which is different between lapack 3.9 and 3.10. >> > > Mathematically, the SVD is not unique, and lapack 3.10 returns a >> > > different (still valid) solution. More precisely, I verified that one >> > > of the right-singular vector of that matrix has the opposite sign in >> > > lapack 3.10. I also verified that the decomposition is correct by >> > > checking that: >> > > >> > > max(abs(barleySVD$u %*% diag(barleySVD$d) %*% t(barleySVD$v) - >> > > barleyMatrix)) >> > > >> > > is a small value (about 2e-14). >> > > >> > > Also note that the hardcoded expected values already partially differ >> > > from those of the original research paper mentioned in that test >> > > (Gabriel (1998): Generalised bilinear regression). More precisely, half >> > > of the values were hardcoded with the opposite sign. It seems that now >> > > all values need to be hardcoded with the opposite sign. >> > > >> > > The testsuite of r-cran-gnm thus needs to be adapted, by being more >> > > tolerant to such sign changes. >> > > >> > > N.B. : when trying to reproduce the problem, please ensure that your >> > > lapack alternative (as given by “update-alternatives --display >> > > liblapack.so.3-x86_64-linux-gnu) points to /usr/lib/x86_64-linux- >> > > gnu/lapack/liblapack.so.3, and not to the binary provided by either >> > > openblas or atlas (because these two have not yet been recompiled >> > > against lapack 3.10, and thus do not expose the problem). >> > > >> > > Best regards, >> > > >> > > -- >> > > ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot >> > > ⣾⠁⢠⠒⠀⣿⡁ Debian Developer >> > > ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name >> > > ⠈⠳⣄ https://www.debian.org >> > > >> > >> > >> > >> > > ___ >> > > R-pkg-team mailing list >> > > r-pkg-t...@alioth-lists.debian.net >> > > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/r-pkg-team >> > >> > >> > -- >> > http://fam-tille.de > > -- > ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot > ⣾⠁⢠⠒⠀⣿⡁ Debian Developer > ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name > ⠈⠳⣄ https://www.debian.org > > > Attachments: > * lapack-3.10.patch > * signature.asc
Bug#994457: r-cran-gnm autopkgtest needs to be adapted for lapack 3.10
Dear Heather,
Please find attached the patch that I am going to apply to r-cran-gnm
in Debian.
With this patch applied, the test now works with both lapack 3.9 and
3.10.
Feel free to use it (or not) for the next release of gnm.
Best wishes,
Le lundi 27 septembre 2021 à 12:01 +0100, Heather Turner a écrit :
> Hi Andreas,
>
> Thanks for forwarding this bug report and thanks to Sébastien for the
> detailed and accurate analysis. I will need to submit an update to CRAN,
> which should be feasible in the next week or two. Is there a deadline that I
> need to work to?
>
> Best wishes,
>
> Heather
>
> On Mon, Sep 27, 2021, at 10:12 AM, Andreas Tille wrote:
> > Control: tags -1 upstream
> > Control: forwarded -1 Heather Turner
> >
> > Hi Heather,
> >
> > the Debian packaged gnm recieved a bug report about a failing test in
> > connection with the upgrade to lapack 3.10.0 on the machine running the
> > test. Please read the bug report below.
> >
> > We admit we need your help to solve this issue that might affect
> > other systems as well.
> >
> > Kind regards
> >
> > Andreas.
> >
> > On Thu, Sep 16, 2021 at 10:59:19AM +0200, Sébastien Villemot wrote:
> > > Package: r-cran-gnm
> > > Version: 1.1-1-2
> > > Severity: serious
> > > Tags: sid bookworm
> > > X-Debbugs-CC: debian...@lists.debian.org
> > > User: debian...@lists.debian.org
> > > Usertags: needs-update
> > >
> > > Dear Maintainer,
> > >
> > > Since the upload of lapack 3.10.0-1, the autopkgtest of r-cran-gnm
> > > fails in unstable. See for example:
> > > https://ci.debian.net/data/autopkgtest/unstable/amd64/r/r-cran-gnm/15155026/log.gz
> > >
> > > More precisely, test-biplot.R fails, because some results have the
> > > opposite sign compared to the one which is expected.
> > >
> > > My understanding is that this comes from the SVD of barleyMatrix in
> > > that test file, which is different between lapack 3.9 and 3.10.
> > > Mathematically, the SVD is not unique, and lapack 3.10 returns a
> > > different (still valid) solution. More precisely, I verified that one
> > > of the right-singular vector of that matrix has the opposite sign in
> > > lapack 3.10. I also verified that the decomposition is correct by
> > > checking that:
> > >
> > > max(abs(barleySVD$u %*% diag(barleySVD$d) %*% t(barleySVD$v) -
> > > barleyMatrix))
> > >
> > > is a small value (about 2e-14).
> > >
> > > Also note that the hardcoded expected values already partially differ
> > > from those of the original research paper mentioned in that test
> > > (Gabriel (1998): Generalised bilinear regression). More precisely, half
> > > of the values were hardcoded with the opposite sign. It seems that now
> > > all values need to be hardcoded with the opposite sign.
> > >
> > > The testsuite of r-cran-gnm thus needs to be adapted, by being more
> > > tolerant to such sign changes.
> > >
> > > N.B. : when trying to reproduce the problem, please ensure that your
> > > lapack alternative (as given by “update-alternatives --display
> > > liblapack.so.3-x86_64-linux-gnu) points to /usr/lib/x86_64-linux-
> > > gnu/lapack/liblapack.so.3, and not to the binary provided by either
> > > openblas or atlas (because these two have not yet been recompiled
> > > against lapack 3.10, and thus do not expose the problem).
> > >
> > > Best regards,
> > >
> > > --
> > > ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot
> > > ⣾⠁⢠⠒⠀⣿⡁ Debian Developer
> > > ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name
> > > ⠈⠳⣄ https://www.debian.org
> > >
> >
> >
> >
> > > ___
> > > R-pkg-team mailing list
> > > r-pkg-t...@alioth-lists.debian.net
> > > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/r-pkg-team
> >
> >
> > --
> > http://fam-tille.de
--
⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot
⣾⠁⢠⠒⠀⣿⡁ Debian Developer
⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name
⠈⠳⣄ https://www.debian.org
Description: Fix test-biplot.R with lapack 3.10
With lapack 3.10, test-biplot.R fails, because some results have the
opposite sign compared to the one which is expected.
.
This comes from the SVD of barleyMatrix in that test file, which is different
between lapack 3.9 and 3.10. Mathematically, the SVD is not unique, and lapack
3.10 returns a different (still valid) solution.
.
This patch accomodates sign differences in the columns of the test matrices,
so that the test passes with both lapack 3.9 and lapack 3.10.
Author: Sébastien Villemot
Bug-Debian: https://bugs.debian.org/994457
Forwarded: no
Last-Update: 2021-10-11
---
This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
--- a/tests/testthat/test-biplot.R
+++ b/tests/testthat/test-biplot.R
@@ -6,6 +6,15 @@ set.seed(1)
# Gabriel, K R (1998). Generalised bilinear regression. Biometrika 85, 689–700.
+expect_equivalent_up_to_column_sign <- function(A, B)
+{
+ ok <- nrow(A) == nrow(B) && ncol(A) == ncol(B)
+ if (ok)
+for (j in 1:ncol(A))
+ ok <- ok && (all(A[,j] == B[,j]) || all(A[,j] ==
Bug#994457: r-cran-gnm autopkgtest needs to be adapted for lapack 3.10
Hi Heather, thanks a lot for your quick response. We do not have any deadline since Debian 11 was "just" released. Kind regards Andreas. On Mon, Sep 27, 2021 at 12:01:20PM +0100, Heather Turner wrote: > Hi Andreas, > > Thanks for forwarding this bug report and thanks to Sébastien for the > detailed and accurate analysis. I will need to submit an update to CRAN, > which should be feasible in the next week or two. Is there a deadline that I > need to work to? > > Best wishes, > > Heather > > On Mon, Sep 27, 2021, at 10:12 AM, Andreas Tille wrote: > > Control: tags -1 upstream > > Control: forwarded -1 Heather Turner > > > > Hi Heather, > > > > the Debian packaged gnm recieved a bug report about a failing test in > > connection with the upgrade to lapack 3.10.0 on the machine running the > > test. Please read the bug report below. > > > > We admit we need your help to solve this issue that might affect > > other systems as well. > > > > Kind regards > > > > Andreas. > > > > On Thu, Sep 16, 2021 at 10:59:19AM +0200, Sébastien Villemot wrote: > >> Package: r-cran-gnm > >> Version: 1.1-1-2 > >> Severity: serious > >> Tags: sid bookworm > >> X-Debbugs-CC: debian...@lists.debian.org > >> User: debian...@lists.debian.org > >> Usertags: needs-update > >> > >> Dear Maintainer, > >> > >> Since the upload of lapack 3.10.0-1, the autopkgtest of r-cran-gnm > >> fails in unstable. See for example: > >> https://ci.debian.net/data/autopkgtest/unstable/amd64/r/r-cran-gnm/15155026/log.gz > >> > >> More precisely, test-biplot.R fails, because some results have the > >> opposite sign compared to the one which is expected. > >> > >> My understanding is that this comes from the SVD of barleyMatrix in > >> that test file, which is different between lapack 3.9 and 3.10. > >> Mathematically, the SVD is not unique, and lapack 3.10 returns a > >> different (still valid) solution. More precisely, I verified that one > >> of the right-singular vector of that matrix has the opposite sign in > >> lapack 3.10. I also verified that the decomposition is correct by > >> checking that: > >> > >> max(abs(barleySVD$u %*% diag(barleySVD$d) %*% t(barleySVD$v) - > >> barleyMatrix)) > >> > >> is a small value (about 2e-14). > >> > >> Also note that the hardcoded expected values already partially differ > >> from those of the original research paper mentioned in that test > >> (Gabriel (1998): Generalised bilinear regression). More precisely, half > >> of the values were hardcoded with the opposite sign. It seems that now > >> all values need to be hardcoded with the opposite sign. > >> > >> The testsuite of r-cran-gnm thus needs to be adapted, by being more > >> tolerant to such sign changes. > >> > >> N.B. : when trying to reproduce the problem, please ensure that your > >> lapack alternative (as given by “update-alternatives --display > >> liblapack.so.3-x86_64-linux-gnu) points to /usr/lib/x86_64-linux- > >> gnu/lapack/liblapack.so.3, and not to the binary provided by either > >> openblas or atlas (because these two have not yet been recompiled > >> against lapack 3.10, and thus do not expose the problem). > >> > >> Best regards, > >> > >> -- > >> ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot > >> ⣾⠁⢠⠒⠀⣿⡁ Debian Developer > >> ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name > >> ⠈⠳⣄ https://www.debian.org > >> > > > > > > > >> ___ > >> R-pkg-team mailing list > >> r-pkg-t...@alioth-lists.debian.net > >> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/r-pkg-team > > > > > > -- > > http://fam-tille.de > -- http://fam-tille.de
Bug#994457: r-cran-gnm autopkgtest needs to be adapted for lapack 3.10
Hi Andreas, Thanks for forwarding this bug report and thanks to Sébastien for the detailed and accurate analysis. I will need to submit an update to CRAN, which should be feasible in the next week or two. Is there a deadline that I need to work to? Best wishes, Heather On Mon, Sep 27, 2021, at 10:12 AM, Andreas Tille wrote: > Control: tags -1 upstream > Control: forwarded -1 Heather Turner > > Hi Heather, > > the Debian packaged gnm recieved a bug report about a failing test in > connection with the upgrade to lapack 3.10.0 on the machine running the > test. Please read the bug report below. > > We admit we need your help to solve this issue that might affect > other systems as well. > > Kind regards > > Andreas. > > On Thu, Sep 16, 2021 at 10:59:19AM +0200, Sébastien Villemot wrote: >> Package: r-cran-gnm >> Version: 1.1-1-2 >> Severity: serious >> Tags: sid bookworm >> X-Debbugs-CC: debian...@lists.debian.org >> User: debian...@lists.debian.org >> Usertags: needs-update >> >> Dear Maintainer, >> >> Since the upload of lapack 3.10.0-1, the autopkgtest of r-cran-gnm >> fails in unstable. See for example: >> https://ci.debian.net/data/autopkgtest/unstable/amd64/r/r-cran-gnm/15155026/log.gz >> >> More precisely, test-biplot.R fails, because some results have the >> opposite sign compared to the one which is expected. >> >> My understanding is that this comes from the SVD of barleyMatrix in >> that test file, which is different between lapack 3.9 and 3.10. >> Mathematically, the SVD is not unique, and lapack 3.10 returns a >> different (still valid) solution. More precisely, I verified that one >> of the right-singular vector of that matrix has the opposite sign in >> lapack 3.10. I also verified that the decomposition is correct by >> checking that: >> >> max(abs(barleySVD$u %*% diag(barleySVD$d) %*% t(barleySVD$v) - >> barleyMatrix)) >> >> is a small value (about 2e-14). >> >> Also note that the hardcoded expected values already partially differ >> from those of the original research paper mentioned in that test >> (Gabriel (1998): Generalised bilinear regression). More precisely, half >> of the values were hardcoded with the opposite sign. It seems that now >> all values need to be hardcoded with the opposite sign. >> >> The testsuite of r-cran-gnm thus needs to be adapted, by being more >> tolerant to such sign changes. >> >> N.B. : when trying to reproduce the problem, please ensure that your >> lapack alternative (as given by “update-alternatives --display >> liblapack.so.3-x86_64-linux-gnu) points to /usr/lib/x86_64-linux- >> gnu/lapack/liblapack.so.3, and not to the binary provided by either >> openblas or atlas (because these two have not yet been recompiled >> against lapack 3.10, and thus do not expose the problem). >> >> Best regards, >> >> -- >> ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot >> ⣾⠁⢠⠒⠀⣿⡁ Debian Developer >> ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name >> ⠈⠳⣄ https://www.debian.org >> > > > >> ___ >> R-pkg-team mailing list >> r-pkg-t...@alioth-lists.debian.net >> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/r-pkg-team > > > -- > http://fam-tille.de
Processed: Re: Bug#994457: r-cran-gnm autopkgtest needs to be adapted for lapack 3.10
Processing control commands: > tags -1 upstream Bug #994457 [r-cran-gnm] r-cran-gnm autopkgtest needs to be adapted for lapack 3.10 Added tag(s) upstream. > forwarded -1 Heather Turner Bug #994457 [r-cran-gnm] r-cran-gnm autopkgtest needs to be adapted for lapack 3.10 Set Bug forwarded-to-address to 'Heather Turner '. -- 994457: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994457 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#994457: r-cran-gnm autopkgtest needs to be adapted for lapack 3.10
Control: tags -1 upstream Control: forwarded -1 Heather Turner Hi Heather, the Debian packaged gnm recieved a bug report about a failing test in connection with the upgrade to lapack 3.10.0 on the machine running the test. Please read the bug report below. We admit we need your help to solve this issue that might affect other systems as well. Kind regards Andreas. On Thu, Sep 16, 2021 at 10:59:19AM +0200, Sébastien Villemot wrote: > Package: r-cran-gnm > Version: 1.1-1-2 > Severity: serious > Tags: sid bookworm > X-Debbugs-CC: debian...@lists.debian.org > User: debian...@lists.debian.org > Usertags: needs-update > > Dear Maintainer, > > Since the upload of lapack 3.10.0-1, the autopkgtest of r-cran-gnm > fails in unstable. See for example: > https://ci.debian.net/data/autopkgtest/unstable/amd64/r/r-cran-gnm/15155026/log.gz > > More precisely, test-biplot.R fails, because some results have the > opposite sign compared to the one which is expected. > > My understanding is that this comes from the SVD of barleyMatrix in > that test file, which is different between lapack 3.9 and 3.10. > Mathematically, the SVD is not unique, and lapack 3.10 returns a > different (still valid) solution. More precisely, I verified that one > of the right-singular vector of that matrix has the opposite sign in > lapack 3.10. I also verified that the decomposition is correct by > checking that: > > max(abs(barleySVD$u %*% diag(barleySVD$d) %*% t(barleySVD$v) - > barleyMatrix)) > > is a small value (about 2e-14). > > Also note that the hardcoded expected values already partially differ > from those of the original research paper mentioned in that test > (Gabriel (1998): Generalised bilinear regression). More precisely, half > of the values were hardcoded with the opposite sign. It seems that now > all values need to be hardcoded with the opposite sign. > > The testsuite of r-cran-gnm thus needs to be adapted, by being more > tolerant to such sign changes. > > N.B. : when trying to reproduce the problem, please ensure that your > lapack alternative (as given by “update-alternatives --display > liblapack.so.3-x86_64-linux-gnu) points to /usr/lib/x86_64-linux- > gnu/lapack/liblapack.so.3, and not to the binary provided by either > openblas or atlas (because these two have not yet been recompiled > against lapack 3.10, and thus do not expose the problem). > > Best regards, > > -- > ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot > ⣾⠁⢠⠒⠀⣿⡁ Debian Developer > ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name > ⠈⠳⣄ https://www.debian.org > > ___ > R-pkg-team mailing list > r-pkg-t...@alioth-lists.debian.net > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/r-pkg-team -- http://fam-tille.de
Processed: Re: Bug#994457: r-cran-gnm autopkgtest needs to be adapted for lapack 3.10
Processing control commands: > tags -1 help Bug #994457 [r-cran-gnm] r-cran-gnm autopkgtest needs to be adapted for lapack 3.10 Added tag(s) help. -- 994457: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994457 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#994457: r-cran-gnm autopkgtest needs to be adapted for lapack 3.10
Control: tags -1 help Hi Sébastien, thanks a lot for the bug report. On Thu, Sep 16, 2021 at 10:59:19AM +0200, Sébastien Villemot wrote: > > The testsuite of r-cran-gnm thus needs to be adapted, by being more > tolerant to such sign changes. Given that you dived into that issue deep enough to possibly know what needs to be done I'd be super happy if you could provide a patch. If you don't have time for it I'd just forward the issue upstream since my time slices for R packages are *way* exhausted after BioConductor transition and *lots* of after-freeze updates which are not yet done and leave several open problems for the moment. Kind regards Andreas. -- http://fam-tille.de
Bug#994457: r-cran-gnm autopkgtest needs to be adapted for lapack 3.10
Package: r-cran-gnm Version: 1.1-1-2 Severity: serious Tags: sid bookworm X-Debbugs-CC: debian...@lists.debian.org User: debian...@lists.debian.org Usertags: needs-update Dear Maintainer, Since the upload of lapack 3.10.0-1, the autopkgtest of r-cran-gnm fails in unstable. See for example: https://ci.debian.net/data/autopkgtest/unstable/amd64/r/r-cran-gnm/15155026/log.gz More precisely, test-biplot.R fails, because some results have the opposite sign compared to the one which is expected. My understanding is that this comes from the SVD of barleyMatrix in that test file, which is different between lapack 3.9 and 3.10. Mathematically, the SVD is not unique, and lapack 3.10 returns a different (still valid) solution. More precisely, I verified that one of the right-singular vector of that matrix has the opposite sign in lapack 3.10. I also verified that the decomposition is correct by checking that: max(abs(barleySVD$u %*% diag(barleySVD$d) %*% t(barleySVD$v) - barleyMatrix)) is a small value (about 2e-14). Also note that the hardcoded expected values already partially differ from those of the original research paper mentioned in that test (Gabriel (1998): Generalised bilinear regression). More precisely, half of the values were hardcoded with the opposite sign. It seems that now all values need to be hardcoded with the opposite sign. The testsuite of r-cran-gnm thus needs to be adapted, by being more tolerant to such sign changes. N.B. : when trying to reproduce the problem, please ensure that your lapack alternative (as given by “update-alternatives --display liblapack.so.3-x86_64-linux-gnu) points to /usr/lib/x86_64-linux- gnu/lapack/liblapack.so.3, and not to the binary provided by either openblas or atlas (because these two have not yet been recompiled against lapack 3.10, and thus do not expose the problem). Best regards, -- ⢀⣴⠾⠻⢶⣦⠀ Sébastien Villemot ⣾⠁⢠⠒⠀⣿⡁ Debian Developer ⢿⡄⠘⠷⠚⠋⠀ https://sebastien.villemot.name ⠈⠳⣄ https://www.debian.org signature.asc Description: This is a digitally signed message part
