Hi Dirk, Do you think it's worth also/instead considering a fix to S4 to avoid this caching issue in future R versions?
(This is top of my for me as we consider the design of S7, and I recently made a note to ensure we avoid similar problems there: https://github.com/RConsortium/OOP-WG/issues/317) Hadley On Sun, Aug 6, 2023 at 4:05 PM Dirk Eddelbuettel <e...@debian.org> wrote: > > > CRAN, by relying on the powerful package management system that is part of R, > provides an unparalleled framework for extending R with nearly 20k packages. > > We recently encountered an issue that highlights a missing element in the > otherwise outstanding package management system. So we would like to start a > discussion about enhancing its feature set. As shown below, a mechanism to > force reinstallation of packages may be needed. > > A demo is included below, it is reproducible in a container. We find the > easiest/fastest reproduction is by saving the code snippet below in the > current directory as eg 'matrixIssue.R' and have it run in a container as > > docker run --rm -ti -v `pwd`:/mnt rocker/r2u Rscript /mnt/matrixIssue.R > > This runs in under two minutes, first installing the older Matrix, next > installs SeuratObject, and then by removing the older Matrix making the > (already installed) current Matrix version the default. This simulates a > package update for Matrix. Which, as the final snippet demonstrates, silently > breaks SeuratObject as the cached S4 method Csparse_validate is now missing. > So when SeuratObject was installed under Matrix 1.5.1, it becomes unuseable > under Matrix 1.6.0. > > What this shows is that a call to update.packages() will silently corrupt an > existing installation. We understand that this was known and addressed at > CRAN by rebuilding all binary packages (for macOS and Windows). > > But it leaves both users relying on source installation as well as > distributors of source packages in a dire situation. It hurt me three times: > my default R installation was affected with unit tests (involving > SeuratObject) silently failing. It similarly broke our CI setup at work. And > it created a fairly bad headache for the Debian packaging I am involved with > (and I surmise it affects other distro similarly). > > It would be good to have a mechanism where a package, when being upgraded, > could flag that 'more actions are required' by the system (administrator). > We think this example demonstrates that we need such a mechanism to avoid > (silently !!) breaking existing installations, possibly by forcing > reinstallation of other packages. R knows the package dependency graph and > could trigger this, possibly after an 'opt-in' variable the user / admin > sets. > > One possibility may be to add a new (versioned) field 'Breaks:'. Matrix could > then have added 'Breaks: SeuratObject (<= 4.1.3)' preventing an installation > of Matrix 1.6.0 when SeuratObject 4.1.3 (or earlier) is present, but > permitting an update to Matrix 1.6.0 alongside a new version, say, 4.1.4 of > SeuratObject which could itself have a versioned Depends: Matrix (>= 1.6.0). > > Regards, Dirk > > > ## Code example follows. Recommended to run the rocker/r2u container. > ## Could also run 'apt update -qq; apt upgrade -y' but not required > ## Thanks to my colleague Paul Hoffman for the core of this example > > ## now have Matrix 1.6.0 because r2u and CRAN remain current but we can > install an older Matrix > remotes::install_version('Matrix', '1.5.1') > > ## we can confirm that we have Matrix 1.5.1 > packageVersion("Matrix") > > ## we now install SeuratObject from source and to speed things up we first > install the binary > install.packages("SeuratObject") # in this container via bspm/r2u as binary > ## and then force a source installation (turning bspm off) _while Matrix is > at 1.5.1_ > if (requireNamespace("bspm", quietly=TRUE) bspm::disable() > Sys.setenv(PKG_CXXFLAGS='-Wno-ignored-attributes') # Eigen compilation > noise silencer > install.packages('SeuratObject') > > ## we now remove the Matrix package version 1.5.1 we installed into > /usr/local leaving 1.6.0 > remove.packages("Matrix") > packageVersion("Matrix") > > ## and we now run a bit of SeuratObject code that is now broken as > Csparse_validate is gone > suppressMessages(library(SeuratObject)) > data('pbmc_small') > graph <- pbmc_small[['RNA_snn']] > class(graph) > getClass('Graph') > show(graph) # this fails > > > -- > dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org > > ______________________________________________ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel -- http://hadley.nz ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
