Re: [Rd] Should c(..., recursive = TRUE) and unlist(x, recursive = TRUE) recurse into expression vectors?
On 2024-04-27 10:53 am, Mikael Jagan wrote:
Reading the body of function 'AnswerType' in bind.c, called from 'do_c'
and 'do_unlist', I notice that EXPRSXP and VECSXP are handled identically
in the recurse = TRUE case.
A corollary is that c(recursive = TRUE) and unlist(recursive = TRUE)
treat expression vectors like expression(a, b) as lists of symbols and
calls. And since they treat symbols and calls as lists of length 1, we
see:
> x <- expression(a, b); y <- expression(c, d)
> c(x, y)
expression(a, b, c, d)
> c(x, y, recursive = TRUE)
[[1]]
a
[[2]]
b
[[3]]
c
[[4]]
d
My expectation based on the documentation in help("c") and help("unlist")
is that those functions would recurse into lists and pairlists, but _not_
into expression vectors.
recursive: logical. If 'recursive = TRUE', the function recursively
descends through lists (and pairlists) combining all their
elements into a vector.
recursive: logical. Should unlisting be applied to list components of
'x'?
My feeling is that either:
(1) the behaviour should change, so that both calls to 'c' above give
the result of type "expression".
(2) the documentation should change to say that expression vectors are
handled as lists in the recursive case.
Option (2) won't break anything but is a bit awkward because it means
that a type "higher" in the documented hierarchy (... < list < expression)
is coerced to a lower type.
Er - this last comment about Option (2) being awkward can be ignored. The
expression vector is not itself coerced to a list. Rather, its non-vector
components are treated as lists of length 1. And that's well-documented.
If anything, Option (1) is awkward as it would treat two types of generic
vectors, list and expression, asymmetrically ...
I can submit a patch implementing Option (2) in a few days to allow for
comments if any.
Mikael
I'll add here that, confusingly, help("expression") says: "an object of
mode 'expression' is a list". I understand the author's intent (lists and
expression vectors differ only in the 'type' field of the SEXP header) but
I wonder if substituting "list" with "generic vector" there would cause
less confusion ... ?
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Should c(..., recursive = TRUE) and unlist(x, recursive = TRUE) recurse into expression vectors?
Reading the body of function 'AnswerType' in bind.c, called from 'do_c'
and 'do_unlist', I notice that EXPRSXP and VECSXP are handled identically
in the recurse = TRUE case.
A corollary is that c(recursive = TRUE) and unlist(recursive = TRUE)
treat expression vectors like expression(a, b) as lists of symbols and
calls. And since they treat symbols and calls as lists of length 1, we
see:
> x <- expression(a, b); y <- expression(c, d)
> c(x, y)
expression(a, b, c, d)
> c(x, y, recursive = TRUE)
[[1]]
a
[[2]]
b
[[3]]
c
[[4]]
d
My expectation based on the documentation in help("c") and help("unlist")
is that those functions would recurse into lists and pairlists, but _not_
into expression vectors.
recursive: logical. If 'recursive = TRUE', the function recursively
descends through lists (and pairlists) combining all their
elements into a vector.
recursive: logical. Should unlisting be applied to list components of
'x'?
My feeling is that either:
(1) the behaviour should change, so that both calls to 'c' above give
the result of type "expression".
(2) the documentation should change to say that expression vectors are
handled as lists in the recursive case.
Option (2) won't break anything but is a bit awkward because it means
that a type "higher" in the documented hierarchy (... < list < expression)
is coerced to a lower type.
I'll add here that, confusingly, help("expression") says: "an object of
mode 'expression' is a list". I understand the author's intent (lists and
expression vectors differ only in the 'type' field of the SEXP header) but
I wonder if substituting "list" with "generic vector" there would cause
less confusion ... ?
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[R-pkg-devel] Matrix 1.7-0 to be released in March with ABI-breaking SuiteSparse update
Dear users and binary repository maintainers,
We are preparing Matrix version 1.7-0 for CRAN submission on March 11, ahead
of the spring release of R version 4.4.0. The only significant change from
Matrix 1.6-5 is an update of the internal SuiteSparse libraries
(5.10.1 -> 7.6.0). Unfortunately, the old and new SuiteSparse versions are
binary incompatible, hence so too will be the old and new Matrix versions.
A corollary is that users and binary repository maintainers switching between
Matrix < 1.7-0 and Matrix >= 1.7-0 must rebuild _from sources_ packages that
link Matrix:
> tools::package_dependencies("Matrix", which = "LinkingTo", reverse =
TRUE)[[1L]]
[1] "ahMLE" "bayesWatch" "cplm"
[4] "GeneralizedWendland" "geostatsp" "irlba"
[7] "lme4""mcmcsae" "OpenMx"
[10] "PRIMME" "PUlasso" "robustlmm"
[13] "spGARCH" "TMB" "bcSeq"
For users, that means doing, e.g.,
install.packages("lme4", type = "source")
but an alternative for Windows and macOS users without the required tools is
oo <- options(repos = "https://cran.r-project.org/;)
install.packages("Matrix")
install.packages("lme4")
options(oo)
where we trust CRAN to provide binaries compatible with the latest Matrix
version (because we notify CRAN upon submission about required rebuilds).
Once other repositories react with rebuilds, they can be used instead of
CRAN.
Our reverse dependency checks (and history, intuition, ...) show that most
problems (caught segfaults in this case) can be traced to a binary incompatible
lme4 and not to one of the other packages linking Matrix. Still, we recommend
rebuilds for all 15 packages.
Maintainers of packages that link Matrix can implement an .onLoad test for
possible binary incompatibility by comparing the value of
if (utils::packageVersion("Matrix") >= "1.6.2")
Matrix::Matrix.Version()[["abi"]]
else numeric_version("0")
at install time and at load time, warning the user if the values differ.
But please do look at the above and not at packageVersion("Matrix") directly,
as the ABI version is incremented less often than the package version.
Mikael {on behalf of citation("Matrix")$author}
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] Matrix 1.6.2+ versus Matrix 1.6.2-
Naras,
Thanks. I'm a bit confused, because Rmosek does not declare Matrix as a
dependency:
> tools::package_dependencies("Rmosek", which = "all")[[1L]]
[1] "pkgbuild"
nor does it contain code needing compilation:
> packageDescription("Rmosek", fields="NeedsCompilation")
[1] "no"
Can you explain the nature of the dependency and how I can reproduce your
output? Is an _external_ library somehow linking Matrix ... ?
Note that the 3 removed entry points were unused by all reverse LinkingTo
on CRAN and BioC at the time that Matrix 1.6-2 was released. We can
suggest replacements (probably off list; I don't know that R-devel is the
right forum) but only if we are able to see the code being compiled ...
Mikael
> Package Rmosek compiles fine using Matrix versions 1.6.2- but not with
> anything beyond Matrix 1.6.2. (FYI, Rmosek provides R interfaces to the
> excellent MOSEK solver; academic licenses are free.)
>
> Here is the error message:
>
> rmsk_obj_matrices.cc:50:9: error: use of undeclared identifier
'Matrix_isclass_Csparse'
> 50 | if (Matrix_isclass_Csparse(val)) {
>| ^
> rmsk_obj_matrices.cc:171:9: error: use of undeclared identifier
'Matrix_isclass_triplet'
>171 | if (Matrix_isclass_triplet(val)) {
>| ^
> rmsk_obj_matrices.cc:225:24: error: use of undeclared identifier
'M_chm_triplet_to_SEXP'
>225 | matrixhandle.protect(M_chm_triplet_to_SEXP(tmp, 0, 0, 0, NULL,
R_NilValue));
>
> These API entry points are no longer in the recent headers. My quick
> examination shows that the first two seem like mostly R API stuff and so
> can be copied over to Rmosek easily but the last one looks more involved
> in my cursory examination.
>
> I was going to let the author of Rmosek know, but I do not see any
> mention of these API entries going away in the NEWS for Matrix.
>
> Would be good to point the author to a suggested approach or even
> provide the beginnings of a fix. Any thoughts, particularly by Matrix
> authors (Martin, Mikael)?
>
> Thanks in advance.
>
> -Naras
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R-4.3 version list.files function could not work correctly in chinese
FWIW, a user on Stack Overflow just reported the same issue with list.files running R 4.3.z on Windows. They do not observe the issue running R-devel, with Tomas' patch (r84960). It is still the case that their file names did not exceed 260 wide characters. https://stackoverflow.com/q/77527167/12685768 Mikael On 2023-08-17 6:00 am, r-devel-requ...@r-project.org wrote: Message: 5 Date: Wed, 16 Aug 2023 16:00:13 +0200 From: Tomas Kalibera To: Ivan Krylov Cc:"r-devel@r-project.org" Subject: Re: [Rd] R-4.3 version list.files function could not work correctly in chinese Message-ID:<21e91609-85b2-103b-8e23-12eadff62...@gmail.com> Content-Type: text/plain; charset="utf-8"; Format="flowed" On 8/16/23 13:22, Ivan Krylov wrote: On Wed, 16 Aug 2023 09:42:09 +0200 Tomas Kalibera wrote: Fixed in R-devel (84960). Please let me know if you see any problem with the fix. Thank you for implementing the fix! I gave 叶月光 the link to the GitHub Action build of the r84960 installer. Thanks and thanks for looking at the change. I'm worried that 叶月光 was seeing FindNextFileA fail for a different reason (all the examples given at the Capital of Statistics forum seemed to use less than 256/4 = 64 characters per file name...), but maybe this won't reappear with the switch to FindNextFileW. If this keeps happening, it might be worth producing a warning when FindNextFileW() fails with an unexpected GetLastError() value. I've added a warning to R-devel when list.files() on Windows stops listing a directory due to an error. There is probably not more we can do unless there is a revised bug report of the original problem. fs::dir_fs() uses NtQueryDirectoryFile() and WideCharToMultiByte() instead of FindNextFileW() and wcstombs(), but maybe this shouldn't matter. In particular, both list.files() and fs::dir_fs() would fail given a file name that cannot be represented in UTF-8 (invalid UTF-16 surrogate pairs?) Right, R only support file names that are valid strings, this assumption is present at many places in the code, so it is fine/consistent to be here as well. The choice of opendir/readdir in R was probably motivated by minimization of platform-specific code. Best Tomas __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [R-pkg-devel] Note to repository maintainers about ABI change in Matrix 1.6-2
On 2023-11-10 5:04 pm, Mikael Jagan wrote:
For repositories != CRAN:
Matrix 1.6-2 was released on Nov 8. Due to an ABI change, repositories
maintaining R package binaries should ensure that binaries for the first
order reverse LinkingTo of Matrix are rebuilt under Matrix 1.6-2:
> db <- available.packages(repos = "https://cran.r-project.org/;)
> tools::package_dependencies("Matrix", db = db, which = "LinkingTo", reverse
=
TRUE)[[1L]]
[1] "ahMLE" "bayesWatch" "cplm"
[4] "GeneralizedWendland" "geostatsp" "hibayes"
[7] "irlba" "lme4""mcmcsae"
[10] "OpenMx" "PRIMME" "robustlmm"
[13] "spGARCH" "TMB"
The following additional packages were found to have stale SClassExtension
objects cached in their namespaces:
MatrixModels, SeuratObject, softImpute
and so their binaries should be rebuilt also.
Mikael
Typically, ABI changes occur only rarely, when Matrix updates the internal
SuiteSparse libraries and _those_ libraries change _their_ ABI. This case
is an exception: we discovered that certain function prototypes in our
headers were inconsistent with corresponding prototypes in the SuiteSparse
headers, and the "consistent" prototypes were not binary compatible.
It is for this reason (and others) that Matrix has begun versioning its ABI,
as I announced in an earlier thread on R-SIG-Mac:
https://stat.ethz.ch/pipermail/r-sig-mac/2023-October/014890.html
That (in addition to future, more proactive announcements on mailing lists)
should allow repository maintainers to adapt sooner going forward.
Mikael
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Note to repository maintainers about ABI change in Matrix 1.6-2
For repositories != CRAN:
Matrix 1.6-2 was released on Nov 8. Due to an ABI change, repositories
maintaining R package binaries should ensure that binaries for the first
order reverse LinkingTo of Matrix are rebuilt under Matrix 1.6-2:
> db <- available.packages(repos = "https://cran.r-project.org/;)
> tools::package_dependencies("Matrix", db = db, which = "LinkingTo", reverse =
TRUE)[[1L]]
[1] "ahMLE" "bayesWatch" "cplm"
[4] "GeneralizedWendland" "geostatsp" "hibayes"
[7] "irlba" "lme4""mcmcsae"
[10] "OpenMx" "PRIMME" "robustlmm"
[13] "spGARCH" "TMB"
Typically, ABI changes occur only rarely, when Matrix updates the internal
SuiteSparse libraries and _those_ libraries change _their_ ABI. This case
is an exception: we discovered that certain function prototypes in our
headers were inconsistent with corresponding prototypes in the SuiteSparse
headers, and the "consistent" prototypes were not binary compatible.
It is for this reason (and others) that Matrix has begun versioning its ABI,
as I announced in an earlier thread on R-SIG-Mac:
https://stat.ethz.ch/pipermail/r-sig-mac/2023-October/014890.html
That (in addition to future, more proactive announcements on mailing lists)
should allow repository maintainers to adapt sooner going forward.
Mikael
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] c(NA, 0+1i) not the same as c(as.complex(NA), 0+1i)?
On 2023-11-09 3:13 am, Martin Maechler wrote:
Mikael Jagan
on Wed, 8 Nov 2023 11:13:18 -0500 writes:
> So, to summarize, the open questions are:
> (1) Should as.complex(NA_character_) give complex(r=NA_real_, i=0)
> instead of NA_complex_?
> (2) Should the first argument in c(NA, x) and c(NA_integer_, x),
> where typeof(x) == "complex", be promoted to complex(r=NA_real_, i=0)
> instead of NA_complex_?
> My opinions:
> (1) No. The imaginary part of the result of parsing the strings "i",
> "+i", and "-i" can be nonzero.
> Consider, e.g., Im(eval(str2lang("0+1i"))) and Im(as.complex("0+1i")).
> If NA_character_ means "a string with unknown content", then we should
> not assume that the string is parsed as a real number.
> (2) Yes. I'd very much like to preserve the identity of c(Im(NA), Im(x))
> and Im(c(NA, x)) for atomic (excluding raw, character) vectors 'x'.
> And while typing this response I noticed the following in current
R-devel and
> current R-patched:
>> 0+1i
> [1] 0+1i
>> 1i
> [1] 0+1i
>> as.complex("0+1i")
> [1] 0+1i
>> as.complex("1i")
> [1] NA
> Warning message:
> NAs introduced by coercion
> That warning seems wrong to me ...
Well, actually, as we now have had the parser accept
1i or 7i, 3.14i etc
I think that it's not the *warning* that is wrong,
but rather the *result* :
Indeed. I realized immediately that I had mistyped, and hoped that readers
would understand what I meant. Yes, as.complex("1i") ought to give 1i and
_not_ NA.
Mikael
Why should as.complex("1i") be different from one of these?
1i
[1] 0+1i
str2lang("1i")
[1] 0+1i
scan(textConnection("1i"), complex())
Read 1 item
[1] 0+1i
> Mikael
> On 2023-11-07 6:00 am, r-devel-requ...@r-project.org wrote:
>>>>>>> Michael Chirico
>>>>>>> on Mon, 6 Nov 2023 23:18:40 -0800 writes:
>> > Thanks Martin. My hang-up was not on what the outcome of
as.complex(NA)
>> > should be, but rather, how I should read code like c(x, y) generally.
Till
>> > now, I have thought of it like 'c(x, y)' is c(as(x, typeof(y)), y)`
when
>> > "type(y) > type(x)". Basically in my mind, "coercion" in R <->
>> > as.(.) (or coerceVector() in C).
>>
>> > So I tracked down the source (which admittedly has been this way for
much
>> > longer than the present discussion) to see what exactly c() is doing
in
>> > this case:
>>
>>
>https://github.com/r-devel/r-svn/blob/71e7480b07767f3b7d5c45a4247959aa4d83d910/src/main/bind.c#L418-L425
>>
>> > And indeed! It's not "coercion" in the sense I just described...
there's a
>> > branch for the 'x == NA_LOGICAL' case to_convert_ to NA_complex_.
>>
>> Yes; "of course" ... still, I did not answer your main question,
>> as you did ask +/- if c() should not get an adjustment to the
>> new as.complex() |--> (Re = NA, Im = 0)
>> behavior.
>>
>> And that is still a valid open question. ... contrary to what I
>> wrote yesterday; sorry for that "answering a different
>> question".
>>
>> Martin
>>
>>
>> > On Mon, Nov 6, 2023 at 3:08 AM Martin
Maechler
>> > wrote:
>>
>> >> >>>>> Michael Chirico
>> >> >>>>> on Sun, 5 Nov 2023 09:41:42 -0800 writes:
>> >>
>> >> > This is another follow-up to the thread from September
>> >> > "Recent changes to as.complex(NA_real_)".
>> >>
>> >> > A test in data.table was broken by the changes for NA
>> >> > coercion to complex; the breakage essentially comes from
>> >>
>> >> > c(NA, 0+1i)
>> >> > # vs
>> >> > c(as.complex(NA), 0+1i)
>> >>
>> >> > The former is the output we tested against; the latter is
>> >> essentially (via
>> >> > coerceVector() in C) what's generated by our data.table::shift()
>> >>
>> >> > However, these are now (r85472) different:
>> >>
>> >> > Im(c(NA, 0+1i))
>> >> &
Re: [Rd] c(NA, 0+1i) not the same as c(as.complex(NA), 0+1i)?
So, to summarize, the open questions are:
(1) Should as.complex(NA_character_) give complex(r=NA_real_, i=0)
instead of NA_complex_?
(2) Should the first argument in c(NA, x) and c(NA_integer_, x),
where typeof(x) == "complex", be promoted to complex(r=NA_real_, i=0)
instead of NA_complex_?
My opinions:
(1) No. The imaginary part of the result of parsing the strings "i",
"+i", and "-i" can be nonzero.
Consider, e.g., Im(eval(str2lang("0+1i"))) and Im(as.complex("0+1i")).
If NA_character_ means "a string with unknown content", then we should
not assume that the string is parsed as a real number.
(2) Yes. I'd very much like to preserve the identity of c(Im(NA), Im(x))
and Im(c(NA, x)) for atomic (excluding raw, character) vectors 'x'.
And while typing this response I noticed the following in current R-devel and
current R-patched:
> 0+1i
[1] 0+1i
> 1i
[1] 0+1i
> as.complex("0+1i")
[1] 0+1i
> as.complex("1i")
[1] NA
Warning message:
NAs introduced by coercion
That warning seems wrong to me ...
Mikael
On 2023-11-07 6:00 am, r-devel-requ...@r-project.org wrote:
Michael Chirico
on Mon, 6 Nov 2023 23:18:40 -0800 writes:
> Thanks Martin. My hang-up was not on what the outcome of as.complex(NA)
> should be, but rather, how I should read code like c(x, y) generally.
Till
> now, I have thought of it like 'c(x, y)' is c(as(x, typeof(y)), y)` when
> "type(y) > type(x)". Basically in my mind, "coercion" in R <->
> as.(.) (or coerceVector() in C).
> So I tracked down the source (which admittedly has been this way for much
> longer than the present discussion) to see what exactly c() is doing in
> this case:
>https://github.com/r-devel/r-svn/blob/71e7480b07767f3b7d5c45a4247959aa4d83d910/src/main/bind.c#L418-L425
> And indeed! It's not "coercion" in the sense I just described... there's
a
> branch for the 'x == NA_LOGICAL' case to_convert_ to NA_complex_.
Yes; "of course" ... still, I did not answer your main question,
as you did ask +/- if c() should not get an adjustment to the
new as.complex() |--> (Re = NA, Im = 0)
behavior.
And that is still a valid open question. ... contrary to what I
wrote yesterday; sorry for that "answering a different
question".
Martin
> On Mon, Nov 6, 2023 at 3:08 AM Martin
Maechler
> wrote:
>> > Michael Chirico
>> > on Sun, 5 Nov 2023 09:41:42 -0800 writes:
>>
>> > This is another follow-up to the thread from September
>> > "Recent changes to as.complex(NA_real_)".
>>
>> > A test in data.table was broken by the changes for NA
>> > coercion to complex; the breakage essentially comes from
>>
>> > c(NA, 0+1i)
>> > # vs
>> > c(as.complex(NA), 0+1i)
>>
>> > The former is the output we tested against; the latter is
>> essentially (via
>> > coerceVector() in C) what's generated by our data.table::shift()
>>
>> > However, these are now (r85472) different:
>>
>> > Im(c(NA, 0+1i))
>> > # [1] NA 1
>> > Im(c(as.complex(NA), 0+1i))
>> > # [1] 0 1
>>
>>
>> > The former matches the behavior of directly using NA_complex_:
>>
>> > Im(c(NA_complex_, 0+1i))
>> > # [1] NA 1
>>
>> > On R4.3.2, they both match the NA_complex_ behavior:
>> > Im(c(NA, 0+1i))
>> > # [1] NA 1
>> > Im(c(as.complex(NA), 0+1i))
>> > # [1] NA 1
>>
>> > Is this intended behavior, does something need to be updated for c()
>> as
>> > well?
>>
>> > Certainly it's messing with my understanding of how c() behaves,
>> e.g. in ?c
>>
>> >> All arguments are coerced to a common type which is the type of the
>> > returned value
>>
>> I think you have confused yourself, and everything behaves as expected:
>>
>> As we now have (in R-devel, since {r85233 | maechler | 2023-09-29 })
>>
>> • ‘as.complex(x)’ now returns ‘complex(real=x, imaginary=0)’
>> for_all_ numerical and logical ‘x’, notably also for ‘NA’
>> or ‘NA_integer_’.
>>
>> ==> as.complex(NA) is indeed complex(real = NA, imaginary = 0)
>>
>> And now, in your
>>
>> c(as.complex(NA), 0+1i)
>>
>> you are calling c() on two complex numbers, i.e., there is*no* coercion
>> (and c(.) is rather "trivial"), and the same is true for
>>
>> c(NA_complex_, 0+1i)
>>
>>
>> However, in 85233, I had only modified & added examples to ?as.complex,
>> and now have added more (corresponding to the above NEWS entry);
-> svn rev 85475
>>
>> .
>>
>> The underlying "dilemma" that nobody can help us with is that
>> "almost infinitely" many different complex numbers z fulfill
>> is.na(z) |--> TRUE
Re: [R-pkg-devel] Matrix and Mac OS
A hack that seems to work is (whitespace added for readability):
\newcommand{\Seqn}{
\ifelse{latex}{
\Sexpr[results=rd]{if (getRversion() < "4.2.2") "eqn{#1}" else
"eqn{#2}"}
}{
\ifelse{html}{
\Sexpr[results=rd]{if (getRversion() < "4.2.0") "eqn{#1}"
else "eqn{#2}"}
}{
\Sexpr[results=rd]{"eqn{#2}"}
}
}
}
as amsmath support for PDF output and KaTeX support for HTML output
were introduced in R 4.2.2 and 4.2.0, respectively.
Sadly I really do seem to need 8 escapes:
\Seqn{text{min}(m,n) times n}{min(m,n)-by-n}
Maybe one of the Rd experts here can suggest an improvement ...
Mikael
On 2023-11-01 5:06 am, Martin Maechler wrote:
Uwe Ligges
on Wed, 1 Nov 2023 06:26:23 +0100 writes:
> On 01.11.2023 03:51, Mikael Jagan wrote:
>> Thanks. It seems that we were mistaken in our feeling (IIRC) that it
would
>> be "OK" to implicitly require '--no-manual' on versions of R from 3.5.0
to
>> 4.2.1, not changing our Depends.
>>
>> We will fix this in Matrix 1.6-2, probably by conditionalizing or
otherwise
>> replacing the amsmath commands and probably _not_ by changing to depend
on
>> R >= 4.2.2. Martin may have more to say in "the morning".
I agree (*not* to raise Matrix pkg's R version dependency).
> Note that dependin on R >= 4.2.2 does not work. We need dependencies of
> the form R >= x.y.0. This is also part of the checks.
Yes, indeed.
And as we learned, R >= 4.2.0 would not help for r-oldrel-macos
I (am unhappy but) agree to take the responsibility for our
decision to go ahead and use much nicer LaTeX formula for
matrices etc, in our help pages {thinking that indeed people who'd
install Matrix on an old R version would always be able to read
Matrix manual pages via web search (as it seems to me 95% of
people do nowadays) ... or then have someone in their
organization to figure out how to use a newer amsmath (latex) package if
they really really want the Matrix pdf manual offline}.
Martin
> Reason is that we have only one binary repository for one R-x.y.?
> series. On WIndows, where we check with R-4.2.3, a binary would be
> created and hence R-4.2.[0-1] would not see any valid Matrix binaries.
> So please either make this work on R >= 4.2.0 or require R >= 4.3.0. If
> the latter, ideally with an interim version that works for R >= 4.2.0,
> so that we valid binaries with correct dependency declarations again.
> Best,
> Uwe
>> In the mean time (i.e., while we are stuck with Matrix 1.6-1.1), it may
>> help
>> to update to R 4.2.3 on r-oldrel-macos-* and/or to have EdSurvey revert
its
>> strict version requirement, unless there are clear examples justifying
one.
>>
>> Mikael
>>
>>
>> On 2023-10-31 8:17 pm, Simon Urbanek wrote:
>>> Mikael,
>>>
>>> in that case I think your requirements are wrong - Matrix says R >=
>>> 3.5.0 which is apparently incorrect - from what you say it should be
>>> 4.2.2?. I can certainly update to 4.2.3 if necessary.
>>>
>>> Cheers,
>>> Simon
>>>
>>>
>>>
>>>> On 1/11/2023, at 9:19 AM, Mikael Jagan wrote:
>>>>
>>>> Thanks. We did see those ERRORs, stemming from use (since Matrix
1.6-0)
>>>> of amsmath commands in Rd files. These have been supported since R
>>>> 4.2.2,
>>>> but r-oldrel-macos-* (unlike r-oldrel-windows-*) continues to run R
>>>> 4.2.0.
>>>> My expectation was that those machines would begin running R >= 4.2.2
>>>> well
>>>> before the R 4.4.0 release, but apparently that was wrong.
>>>>
>>>> I am hesitant to complicate our Rd files with conditions on R versions
>>>> only to support PDF output for R < 4.2.2, but maybe we can consider it
>>>> for the Matrix 1.6-2 release if it is really a barrier for others ...
>>>>
>>>> Mikael
>>>>
>>>> On 2023-10-31 3:33 pm, Simon Urbanek wrote:
>>>>> Mikael,
>>>>> current Matrix fails checks on R-oldrel so that's why only the last
>>>>> working version is installed:
>>>>> https://cran.r-project.org/web/checks/check_results_Matrix.html
>>>>&g
Re: [R-pkg-devel] Matrix and Mac OS
On 2023-11-01 12:59 pm, Mikael Jagan wrote:
A hack that seems to work is (whitespace added for readability):
\newcommand{\Seqn}{
\ifelse{latex}{
\Sexpr[results=rd]{if (getRversion() < "4.2.2") "eqn{#1}" else
"eqn{#2}"}
}{
\ifelse{html}{
\Sexpr[results=rd]{if (getRversion() < "4.2.0") "eqn{#1}"
else "eqn{#2}"}
}{
\Sexpr[results=rd]{"eqn{#2}"}
}
}
}
Er, the above is wrong, because '<' should be '>=' and because '#2' (which
is conceptually verbatim text) should use \verb{} for PDF and HTML output,
not \eqn{}. For Matrix 1.6-2 I have created man/macros/local.Rd and added:
\newcommand{\Seqn}{\ifelse{latex}{\Sexpr[results=rd]{if (getRversion() >=
"4.2.2") "eqn{#1}" else
"verb{#2}"}}{\ifelse{html}{\Sexpr[results=rd]{if (getRversion() >=
"4.2.0") "eqn{#1}" else
"verb{#2}"}}{\Sexpr[results=rd]{"eqn{#2}"
\newcommand{\Sdeqn}{\ifelse{latex}{\Sexpr[results=rd]{if (getRversion() >=
"4.2.2") "deqn{#1}" else
"preformatted{#2}"}}{\ifelse{html}{\Sexpr[results=rd]{if (getRversion()
>= "4.2.0") "deqn{#1}" else
"preformatted{#2}"}}{\Sexpr[results=rd]{"deqn{#2}"
Now Matrix 1.6-2 passes its Rd checks under my checkout of R-3-5-branch.
Some examples and tests fail for unrelated reasons. I'll fix those, too ...
Mikael
as amsmath support for PDF output and KaTeX support for HTML output
were introduced in R 4.2.2 and 4.2.0, respectively.
Sadly I really do seem to need 8 escapes:
\Seqn{text{min}(m,n) times n}{min(m,n)-by-n}
Maybe one of the Rd experts here can suggest an improvement ...
Mikael
On 2023-11-01 5:06 am, Martin Maechler wrote:
Uwe Ligges
on Wed, 1 Nov 2023 06:26:23 +0100 writes:
> On 01.11.2023 03:51, Mikael Jagan wrote:
>> Thanks. It seems that we were mistaken in our feeling (IIRC) that it
would
>> be "OK" to implicitly require '--no-manual' on versions of R from
3.5.0 to
>> 4.2.1, not changing our Depends.
>>
>> We will fix this in Matrix 1.6-2, probably by conditionalizing or
otherwise
>> replacing the amsmath commands and probably _not_ by changing to
depend on
>> R >= 4.2.2. Martin may have more to say in "the morning".
I agree (*not* to raise Matrix pkg's R version dependency).
> Note that dependin on R >= 4.2.2 does not work. We need dependencies of
> the form R >= x.y.0. This is also part of the checks.
Yes, indeed.
And as we learned, R >= 4.2.0 would not help for r-oldrel-macos
I (am unhappy but) agree to take the responsibility for our
decision to go ahead and use much nicer LaTeX formula for
matrices etc, in our help pages {thinking that indeed people who'd
install Matrix on an old R version would always be able to read
Matrix manual pages via web search (as it seems to me 95% of
people do nowadays) ... or then have someone in their
organization to figure out how to use a newer amsmath (latex) package if
they really really want the Matrix pdf manual offline}.
Martin
> Reason is that we have only one binary repository for one R-x.y.?
> series. On WIndows, where we check with R-4.2.3, a binary would be
> created and hence R-4.2.[0-1] would not see any valid Matrix binaries.
> So please either make this work on R >= 4.2.0 or require R >= 4.3.0. If
> the latter, ideally with an interim version that works for R >= 4.2.0,
> so that we valid binaries with correct dependency declarations again.
> Best,
> Uwe
>> In the mean time (i.e., while we are stuck with Matrix 1.6-1.1), it may
>> help
>> to update to R 4.2.3 on r-oldrel-macos-* and/or to have EdSurvey
revert its
>> strict version requirement, unless there are clear examples justifying
one.
>>
>> Mikael
>>
>>
>> On 2023-10-31 8:17 pm, Simon Urbanek wrote:
>>> Mikael,
>>>
>>> in that case I think your requirements are wrong - Matrix says R >=
>>> 3.5.0 which is apparently incorrect - from what you say it should be
>>> 4.2.2?. I can certainly update to 4.2.3 if necessary.
>>>
>>> Cheers,
>>> Simon
>>>
>>>
>>>
>>>> On 1/11/2023, at 9:19 AM, Mikael Jagan wrote:
>>>>
>>>> Thanks. We d
Re: [R-pkg-devel] Matrix and Mac OS
Thanks. It seems that we were mistaken in our feeling (IIRC) that it would be "OK" to implicitly require '--no-manual' on versions of R from 3.5.0 to 4.2.1, not changing our Depends. We will fix this in Matrix 1.6-2, probably by conditionalizing or otherwise replacing the amsmath commands and probably _not_ by changing to depend on R >= 4.2.2. Martin may have more to say in "the morning". In the mean time (i.e., while we are stuck with Matrix 1.6-1.1), it may help to update to R 4.2.3 on r-oldrel-macos-* and/or to have EdSurvey revert its strict version requirement, unless there are clear examples justifying one. Mikael On 2023-10-31 8:17 pm, Simon Urbanek wrote: Mikael, in that case I think your requirements are wrong - Matrix says R >= 3.5.0 which is apparently incorrect - from what you say it should be 4.2.2?. I can certainly update to 4.2.3 if necessary. Cheers, Simon On 1/11/2023, at 9:19 AM, Mikael Jagan wrote: Thanks. We did see those ERRORs, stemming from use (since Matrix 1.6-0) of amsmath commands in Rd files. These have been supported since R 4.2.2, but r-oldrel-macos-* (unlike r-oldrel-windows-*) continues to run R 4.2.0. My expectation was that those machines would begin running R >= 4.2.2 well before the R 4.4.0 release, but apparently that was wrong. I am hesitant to complicate our Rd files with conditions on R versions only to support PDF output for R < 4.2.2, but maybe we can consider it for the Matrix 1.6-2 release if it is really a barrier for others ... Mikael On 2023-10-31 3:33 pm, Simon Urbanek wrote: Mikael, current Matrix fails checks on R-oldrel so that's why only the last working version is installed: https://cran.r-project.org/web/checks/check_results_Matrix.html Cheers, Simon On 1/11/2023, at 4:05 AM, Mikael Jagan wrote: I am guessing that they mean EdSurvey: https://cran.r-project.org/web/checks/check_results_EdSurvey.html Probably Matrix 1.6-1.1 is not installed on r-oldrel-macos-arm64, even though it can be, because it was not released until R 4.3-z. AFAIK, methods for 'qr' have not been touched since Matrix 1.6-0, and even those changes should have been backwards compatible, modulo handling of dimnames (class sparseQR gained a Dimnames slot in 1.6-0). So I don't see a clear reason for requiring 1.6-1.1. Requiring 1.6-0 might make sense, if somehow EdSurvey depends on how class sparseQR preserves dimnames. But IIRC our rev. dep. checks at that time did not reveal problems with EdSurvey. Mikael On 2023-10-31 7:00 am, r-package-devel-requ...@r-project.org wrote: Paul, can you give us a bit more detail? Which package, which build and where you got the errors? Older builds may not have the latest Matrix. Cheers, Simon On 31/10/2023, at 11:26 AM, Bailey, Paul via R-package-devel wrote: Hi, I'm the maintainer for a few packages, one of which is currently failing CRAN checks on Mac OS because Matrix is not available in my required version (the latest). I had to fix a few things due to changes in the latest Matrix package because of how qr works and I thought, given the apparent API change, I should then require the latest version. My error is, "Package required and available but unsuitable version: 'Matrix'" When I look at the NEWS in Matrix there is no mention of Mac OS issues, what the latest stable version of Matrix is, nor when a fix is expected. What version do MacOS version test Matrix with by default? Where is this documented? I assumes it always tested with the latest version on CRAN, so I'm a bit surprised. Or will this be resolved soon and I shouldn't bother CRAN maintainers with a new version of my package? Best, Paul [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Matrix and Mac OS
Thanks. We did see those ERRORs, stemming from use (since Matrix 1.6-0) of amsmath commands in Rd files. These have been supported since R 4.2.2, but r-oldrel-macos-* (unlike r-oldrel-windows-*) continues to run R 4.2.0. My expectation was that those machines would begin running R >= 4.2.2 well before the R 4.4.0 release, but apparently that was wrong. I am hesitant to complicate our Rd files with conditions on R versions only to support PDF output for R < 4.2.2, but maybe we can consider it for the Matrix 1.6-2 release if it is really a barrier for others ... Mikael On 2023-10-31 3:33 pm, Simon Urbanek wrote: Mikael, current Matrix fails checks on R-oldrel so that's why only the last working version is installed: https://cran.r-project.org/web/checks/check_results_Matrix.html Cheers, Simon On 1/11/2023, at 4:05 AM, Mikael Jagan wrote: I am guessing that they mean EdSurvey: https://cran.r-project.org/web/checks/check_results_EdSurvey.html Probably Matrix 1.6-1.1 is not installed on r-oldrel-macos-arm64, even though it can be, because it was not released until R 4.3-z. AFAIK, methods for 'qr' have not been touched since Matrix 1.6-0, and even those changes should have been backwards compatible, modulo handling of dimnames (class sparseQR gained a Dimnames slot in 1.6-0). So I don't see a clear reason for requiring 1.6-1.1. Requiring 1.6-0 might make sense, if somehow EdSurvey depends on how class sparseQR preserves dimnames. But IIRC our rev. dep. checks at that time did not reveal problems with EdSurvey. Mikael On 2023-10-31 7:00 am, r-package-devel-requ...@r-project.org wrote: Paul, can you give us a bit more detail? Which package, which build and where you got the errors? Older builds may not have the latest Matrix. Cheers, Simon On 31/10/2023, at 11:26 AM, Bailey, Paul via R-package-devel wrote: Hi, I'm the maintainer for a few packages, one of which is currently failing CRAN checks on Mac OS because Matrix is not available in my required version (the latest). I had to fix a few things due to changes in the latest Matrix package because of how qr works and I thought, given the apparent API change, I should then require the latest version. My error is, "Package required and available but unsuitable version: 'Matrix'" When I look at the NEWS in Matrix there is no mention of Mac OS issues, what the latest stable version of Matrix is, nor when a fix is expected. What version do MacOS version test Matrix with by default? Where is this documented? I assumes it always tested with the latest version on CRAN, so I'm a bit surprised. Or will this be resolved soon and I shouldn't bother CRAN maintainers with a new version of my package? Best, Paul [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Matrix and Mac OS
Re-sending to the list with correct subject line ... I should undigest myself ... Mikael On 2023-10-31 11:05 am, Mikael Jagan wrote: I am guessing that they mean EdSurvey: https://cran.r-project.org/web/checks/check_results_EdSurvey.html Probably Matrix 1.6-1.1 is not installed on r-oldrel-macos-arm64, even though it can be, because it was not released until R 4.3-z. AFAIK, methods for 'qr' have not been touched since Matrix 1.6-0, and even those changes should have been backwards compatible, modulo handling of dimnames (class sparseQR gained a Dimnames slot in 1.6-0). So I don't see a clear reason for requiring 1.6-1.1. Requiring 1.6-0 might make sense, if somehow EdSurvey depends on how class sparseQR preserves dimnames. But IIRC our rev. dep. checks at that time did not reveal problems with EdSurvey. Mikael On 2023-10-31 7:00 am, r-package-devel-requ...@r-project.org wrote: Paul, can you give us a bit more detail? Which package, which build and where you got the errors? Older builds may not have the latest Matrix. Cheers, Simon On 31/10/2023, at 11:26 AM, Bailey, Paul via R-package-devel wrote: Hi, I'm the maintainer for a few packages, one of which is currently failing CRAN checks on Mac OS because Matrix is not available in my required version (the latest). I had to fix a few things due to changes in the latest Matrix package because of how qr works and I thought, given the apparent API change, I should then require the latest version. My error is, "Package required and available but unsuitable version: 'Matrix'" When I look at the NEWS in Matrix there is no mention of Mac OS issues, what the latest stable version of Matrix is, nor when a fix is expected. What version do MacOS version test Matrix with by default? Where is this documented? I assumes it always tested with the latest version on CRAN, so I'm a bit surprised. Or will this be resolved soon and I shouldn't bother CRAN maintainers with a new version of my package? Best, Paul [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] R-package-devel Digest, Vol 102, Issue 27
I am guessing that they mean EdSurvey: https://cran.r-project.org/web/checks/check_results_EdSurvey.html Probably Matrix 1.6-1.1 is not installed on r-oldrel-macos-arm64, even though it can be, because it was not released until R 4.3-z. AFAIK, methods for 'qr' have not been touched since Matrix 1.6-0, and even those changes should have been backwards compatible, modulo handling of dimnames (class sparseQR gained a Dimnames slot in 1.6-0). So I don't see a clear reason for requiring 1.6-1.1. Requiring 1.6-0 might make sense, if somehow EdSurvey depends on how class sparseQR preserves dimnames. But IIRC our rev. dep. checks at that time did not reveal problems with EdSurvey. Mikael On 2023-10-31 7:00 am, r-package-devel-requ...@r-project.org wrote: Paul, can you give us a bit more detail? Which package, which build and where you got the errors? Older builds may not have the latest Matrix. Cheers, Simon On 31/10/2023, at 11:26 AM, Bailey, Paul via R-package-devel wrote: Hi, I'm the maintainer for a few packages, one of which is currently failing CRAN checks on Mac OS because Matrix is not available in my required version (the latest). I had to fix a few things due to changes in the latest Matrix package because of how qr works and I thought, given the apparent API change, I should then require the latest version. My error is, "Package required and available but unsuitable version: 'Matrix'" When I look at the NEWS in Matrix there is no mention of Mac OS issues, what the latest stable version of Matrix is, nor when a fix is expected. What version do MacOS version test Matrix with by default? Where is this documented? I assumes it always tested with the latest version on CRAN, so I'm a bit surprised. Or will this be resolved soon and I shouldn't bother CRAN maintainers with a new version of my package? Best, Paul [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Question regarding listing base and recommended packages programmatically and efficiently
Maybe something like this:
> isRecommendedPkg <- utils:::isBasePkg
> body(isRecommendedPkg)[[c(3L, 3L)]] <- "recommended"
> installed <- unique(list.files(.libPaths()))
> installed[vapply(installed, isRecommendedPkg, NA)]
[1] "KernSmooth" "MASS" "Matrix" "boot" "class"
[6] "cluster""codetools" "foreign""lattice""mgcv"
[11] "nlme" "nnet" "rpart" "spatial""survival"
where in your package you would define isRecommendedPkg "manually".
Another (but quite undocumented and so maybe not "recommended" :-))
possibility is this:
> mk <- file.path(R.home("share"), "make", "vars.mk")
> pp <- sub("^.*= +", "", grep("^R_PKGS_RECOMMENDED", readLines(mk), value
= TRUE))
> sort(strsplit(pp, " ")[[1L]])
[1] "KernSmooth" "MASS" "Matrix" "boot" "class"
[6] "cluster""codetools" "foreign""lattice""mgcv"
[11] "nlme" "nnet" "rpart" "spatial""survival"
I grepped around and did not find variables in any base namespace containing
the names of these packages. It wouldn't be too hard to define such variables
when R is configured/built, but maybe there are "reasons" to not do that ... ?
Mikael
> It would be much faster (but slightly less reliable) to use
> list.files(.libPaths()) to get the names of all installed packages, and
> then filter them to the known list of base and recommended packages,
> which changes very rarely.
>
> Duncan Murdoch
>
> On 12/10/2023 8:34 a.m., Tony Wilkes wrote:
> > Dear all,
> >
> > In my R package that I'm developing, I use `installed.packages(priority =
"base")` to programmatically get all core/base R packages (i.e. base, stats,
etc.). And similarly, I use installed.packages(priority = "recommended")` to
programmatically get the recommended R packages (i.e. mgcv, lattice, etc.).
> >
> > However, CRAN has requested to not use `installed.packages()`, as it is
slow. I fully get and agree with that assesment. I used installed.packages()`
anyway because I could not find a better, fool-proof alternative.
> >
> > Nonetheless, I was asked to change this code for optimalisation. So I would
like to ask: how do I programmatically get all base/core R packages safely and
efficiently, but without using `installed.packages()`? And the same question for
recommended R packages. I have of course Googled it, and looked at R's
documentation (though R's documentation is large, so it's easy to miss
something); no solution found. So if any of you has a smart idea: I'm all ears.
> >
> > Thank you in advance.
> >
> > Kind regards,
> >
> > Tony.
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Note: no function found corresponding to methods exports for show
On 2023-10-08 5:43 pm, Mikael Jagan wrote:
Hi,
I recently submitted my first R package to CRAN, and I encountered a NOTE
during the checking process. The NOTE looks like this:
=
Flavor: r-devel-linux-x86_64-debian-gcc, r-devel-windows-x86_64
Check: whether the namespace can be loaded with stated dependencies,
Result: NOTE
Warning: no function found corresponding to methods exports from 'hmsr'
for: 'show'
A namespace must be able to be loaded with just the base namespace
loaded: otherwise if the namespace gets loaded by a saved object, the
session will be unable to start.
Probably some imports need to be declared in the NAMESPACE file.
Sounds like your NAMESPACE is missing importFrom(methods, show) ... ?
I'm not sure why you could not reproduce the problem locally, given that
check uses _R_CHECK_CODE_USAGE_WITH_ONLY_BASE_ATTACHED_=true by default.
Reading tools:::.check_packages I see that this environment variable is
not relevant to this particular test. Still, do import generic functions
for which you define methods, from all packages != "base".
Mikael
But it is hard to say anything without a link to the package sources ...
Mikael
=
I'm seeking help to understand and address this issue. show is defined like
this:
#' Show method for class "hms".#'#' @param object - hms s4 object#'#'
@return It returns the names of the slots and the classes associated
with the#' slots in the "hms" class. It prints call details.#'#'
@export#'#' @examples
#' f <- function(x) x
#' result <- hms(fitness = f, lower = -5, upper = 5)
#' show(result)
setMethod("show", "hms", function(object) {
cat("An object of class \"hms\"\n")
cat("\nCall:\n", deparse(object@call), "\n\n", sep = "")
cat("Available slots:\n")
print(methods::slotNames(object))})
And in the NAMESPACE there is a line that looks like this:
exportMethods(show)
I was investigating if the show is defined correctly and the GA package has
a very similar method:
setMethod("show", "ga",function(object)
{ cat("An object of class \"ga\"\n")
cat("\nCall:\n", deparse(object@call), "\n\n",sep="")
cat("Available slots:\n")
print(slotNames(object))})
I could not reproduce this NOTE on my personal machine with Windows.
On Mac OS I have never experienced this NOTE.
Best regards
Wojciech Achtelik
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Note: no function found corresponding to methods exports for show
Hi,
I recently submitted my first R package to CRAN, and I encountered a NOTE
during the checking process. The NOTE looks like this:
=
Flavor: r-devel-linux-x86_64-debian-gcc, r-devel-windows-x86_64
Check: whether the namespace can be loaded with stated dependencies,
Result: NOTE
Warning: no function found corresponding to methods exports from 'hmsr'
for: 'show'
A namespace must be able to be loaded with just the base namespace
loaded: otherwise if the namespace gets loaded by a saved object, the
session will be unable to start.
Probably some imports need to be declared in the NAMESPACE file.
Sounds like your NAMESPACE is missing importFrom(methods, show) ... ?
I'm not sure why you could not reproduce the problem locally, given that
check uses _R_CHECK_CODE_USAGE_WITH_ONLY_BASE_ATTACHED_=true by default.
But it is hard to say anything without a link to the package sources ...
Mikael
=
I'm seeking help to understand and address this issue. show is defined like
this:
#' Show method for class "hms".#'#' @param object - hms s4 object#'#'
@return It returns the names of the slots and the classes associated
with the#' slots in the "hms" class. It prints call details.#'#'
@export#'#' @examples
#' f <- function(x) x
#' result <- hms(fitness = f, lower = -5, upper = 5)
#' show(result)
setMethod("show", "hms", function(object) {
cat("An object of class \"hms\"\n")
cat("\nCall:\n", deparse(object@call), "\n\n", sep = "")
cat("Available slots:\n")
print(methods::slotNames(object))})
And in the NAMESPACE there is a line that looks like this:
exportMethods(show)
I was investigating if the show is defined correctly and the GA package has
a very similar method:
setMethod("show", "ga",function(object)
{ cat("An object of class \"ga\"\n")
cat("\nCall:\n", deparse(object@call), "\n\n",sep="")
cat("Available slots:\n")
print(slotNames(object))})
I could not reproduce this NOTE on my personal machine with Windows.
On Mac OS I have never experienced this NOTE.
Best regards
Wojciech Achtelik
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [Rd] as(, "dgTMatrix")' is deprecated.
> There is a call to mosek and I assumed that this wasn’t going to be helpful
for most R-devel recipients. I tried Duncan’s very reasonable suggestion about
options() but it didn’t produce the desired error, so perhaps this isn’t really
a warning but something else???
>
The details are described in news(package="Matrix") under "Changes in version
1.5-0" and implemented in Matrix:::Matrix.DeprecatedCoerce. That version was
released in Sep 2022.
At the time, it was too disruptive to signal a proper deprecation warning
with every as(., "d(g.|.C)Matrix") call, for every reverse dependency of
Matrix, so we decided that in those most common cases we would use message()
instead of warning(). A corollary was that options(warn=n) would not have
the usual effect, so we advertised an alternative to affected maintainers:
in R:
options(Matrix.warnDeprecatedCoerce=n)
on the command line:
R_MATRIX_WARN_DEPRECATED_COERCE=n R CMD check *.tar.gz
But maybe it is time to begin signaling warnings unconditionally ...
Mikael
> For those who might have rmosek installed, I’m doing:
>
> library(REBayes)
> demo(GLmix1)
>
> Thanks,
> Roger
>
>
> > On Oct 3, 2023, at 6:17 PM, Martin Maechler stat.math.ethz.ch> wrote:
> >
> >> Duncan Murdoch
> >>on Tue, 3 Oct 2023 12:59:10 -0400 writes:
> >
> >> On 03/10/2023 12:50 p.m., Koenker, Roger W wrote:
> >>> I’ve been getting this warning for a while now (about
> >>> five years if memory serves) and I’m finally tired of it,
> >>> but also too tired to track it down in Matrix. As far as
> >>> I can grep I have no reference to either deprecated
> >>> object, only the apparently innocuous Matrix::Matrix(A,
> >>> sparse = TRUE). Can someone advise, Martin perhaps? I
> >>> thought it might come from Rmosek, but mosek folks don’t
> >>> think so.
> >>>
https://urldefense.com/v3/__https://groups.google.com/g/mosek/c/yEwXmMfHBbg/m/l_mkeM4vAAAJ__;!!DZ3fjg!71re8ipw9fFStkMab0wGuPNSzSaAhPI5vwxd1BCQ7a55mYiRpAq2prn9-wREqKL_G2uBYboXISQfxZYCZ9AFxCnwxdzqTw$
> >
> >> A quick scan of that discussion didn't turn up anything
> >> relevant, e.g. a script to produce the warning. Could you
> >> be more specific, or just post the script here?
> >
> >> In general, a good way to locate the source of a warning
> >> is to set options(warn=2) to turn it into an error, and
> >> then trigger it. The traceback from the error will
> >> include a bunch of junk from the code that catches the
> >> warning, but it will also include the context where it was
> >> triggered.
> >
> >> Duncan Murdoch
> >
> > Indeed.
> >
> > But Roger is right that it in the end, (almost surely) it is
> > from our {Matrix} package.
> >
> > Indeed for several years now, we have tried to make the setup
> > leaner (and hence faster) by not explicitly define coercion
> > from to because the size of
> > is here about 200, and we don't want to have to provide
> > 200^2 = 40'000 coercion methods.
> >
> > Rather, Matrix package users should use to high level abstract Matrix
> > classes such as "sparseMatrix" or "CsparseMatrix" or
> > "TsparseMatrix" or "dMatrix", "symmetricMatrix".
> >
> > In the case of as(, "dgTMatrix") , if you
> > replace "dgTMatrix" by "TsparseMatrix"
> > the result will be the same but also work in the future when the
> > deprecation may have been turned into a defunctation ...
> >
> > Martin
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Recent changes to as.complex(NA_real_)
On 2023-09-23 9:43 am, Martin Maechler wrote: Hervé Pagès on Fri, 22 Sep 2023 16:55:05 -0700 writes: > The problem is that you have things that are > **semantically** different but look exactly the same: > They look the same: >> x > [1] NA >> y > [1] NA >> z > [1] NA >> is.na(x) > [1] TRUE >> is.na(y) > [1] TRUE >> is.na(z) > [1] TRUE >> str(x) > cplx NA >> str(y) > num NA >> str(z) > cplx NA > but they are semantically different e.g. >> Re(x) > [1] NA >> Re(y) > [1] -0.5 # surprise! >> Im(x) # surprise! > [1] 2 >> Im(z) > [1] NA > so any expression involving Re() or Im() will produce > different results on input that look the same on the > surface. > You can address this either by normalizing the internal > representation of complex NA to always be complex(r=NaN, > i=NA_real_), like for NA_complex_, or by allowing the > infinite variations that are currently allowed and at the > same time making sure that both Re() and Im() always > return NA_real_ on a complex NA. > My point is that the behavior of complex NA should be > predictable. Right now it's not. Once it's predictable > (with Re() and Im() both returning NA_real_ regardless of > internal representation), then it no longer matters what > kind of complex NA is returned by as.complex(NA_real_), > because they are no onger distinguishable. > H. > On 9/22/23 13:43, Duncan Murdoch wrote: >> Since the result of is.na(x) is the same on each of >> those, I don't see a problem. As long as that is >> consistent, I don't see a problem. You shouldn't be using >> any other test for NA-ness. You should never be >> expecting identical() to treat different types as the >> same (e.g. identical(NA, NA_real_) is FALSE, as it >> should be). If you are using a different test, that's >> user error. >> >> Duncan Murdoch >> >> On 22/09/2023 2:41 p.m., Hervé Pagès wrote: >>> We could also question the value of having an infinite >>> number of NA representations in the complex space. For >>> example all these complex values are displayed the same >>> way (as NA), are considered NAs by is.na(), but are not >>> identical or semantically equivalent (from an Re() or >>> Im() point of view): >>> >>> NA_real_ + 0i >>> >>> complex(r=NA_real_, i=Inf) >>> >>> complex(r=2, i=NA_real_) >>> >>> complex(r=NaN, i=NA_real_) >>> >>> In other words, using a single representation for >>> complex NA (i.e. complex(r=NA_real_, i=NA_real_)) would >>> avoid a lot of unnecessary complications and surprises. >>> >>> Once you do that, whether as.complex(NA_real_) should >>> return complex(r=NA_real_, i=0) or complex(r=NA_real_, >>> i=NA_real_) becomes a moot point. >>> >>> Best, >>> >>> H. Thank you, Hervé. Your proposition is yet another one, to declare that all complex NA's should be treated as identical (almost/fully?) everywhere. This would be a possibility, but I think a drastic one. I think there are too many cases, where I want to keep the information of the real part independent of the values of the imaginary part (e.g. think of the Riemann hypothesis), and typically vice versa. With your proposal, for a (potentially large) vector of complex numbers, after Re(z) <- 1/2 I could no longer rely on Re(z) == 1/2, because it would be wrong for those z where (the imaginary part/ the number) was NA/NaN. Also, in a similar case, a Im(z) <- NA would have to "destroy" all real parts Re(z); not really typically in memory, but effectively for the user, Re(z) would be all NA/NaN. And I think there are quite a few other situations where looking at Re() and Im() separately makes a lot of sense. Indeed, and there is no way to "tell" BLAS and LAPACK to treat both the real and imaginary parts as NA_REAL when either is NA_REAL. Hence the only reliable way to implement such a proposal would be to post-process the result of any computation returning a complex type, testing for NA_REAL and setting both parts to NA_REAL in that case. My expectation is that such testing would drastically slow down basic
Re: [Rd] Recent changes to as.complex(NA_real_)
On 2023-09-22 6:38 am, Martin Maechler wrote:
Mikael Jagan
on Thu, 21 Sep 2023 00:47:39 -0400 writes:
> Revisiting this thread from April:
> https://stat.ethz.ch/pipermail/r-devel/2023-April/082545.html
> where the decision (not yet backported) was made for
> as.complex(NA_real_) to give NA_complex_ instead of
> complex(r=NA_real_, i=0), to be consistent with
> help("as.complex") and as.complex(NA) and as.complex(NA_integer_).
> Was any consideration given to the alternative?
> That is, to changing as.complex(NA) and as.complex(NA_integer_) to
> give complex(r=NA_real_, i=0), consistent with
> as.complex(NA_real_), then amending help("as.complex")
> accordingly?
Hmm, as, from R-core, mostly I was involved, I admit to say "no",
to my knowledge the (above) alternative wasn't considered.
> The principle that
> Im(as.complex()) should be zero
> is quite fundamental, in my view, hence the "new" behaviour
> seems to really violate the principle of least surprise ...
of course "least surprise" is somewhat subjective. Still,
I clearly agree that the above would be one desirable property.
I think that any solution will lead to *some* surprise for some
cases, I think primarily because there are *many* different
values z for which is.na(z) is true, and in any case
NA_complex_ is only of the many.
I also agree with Mikael that we should reconsider the issue
that was raised by Davis Vaughan here ("on R-devel") last April.
> Another (but maybe weaker) argument is that
> double->complex coercions happen more often than
> logical->complex and integer->complex ones. Changing the
> behaviour of the more frequently performed coercion is
> more likely to affect code "out there".
> Yet another argument is that one expects
> identical(as.complex(NA_real_), NA_real_ + (0+0i))
> to be TRUE, i.e., that coercing from double to complex is
> equivalent to adding a complex zero. The new behaviour
> makes the above FALSE, since NA_real_ + (0+0i) gives
> complex(r=NA_real_, i=0).
No! --- To my own surprise (!) --- in current R-devel the above is TRUE,
and
NA_real_ + (0+0i) , the same as
NA_real_ + 0i , really gives complex(r=NA, i=NA) :
Thank you for the correction - indeed, as.complex(NA_real_) and
NA_real_ + (0+0i) are identical in both R-patched and R-devel,
both giving complex(r=NA_real_, i=0) in R-patched and both giving
NA_complex_ in R-devel. I was hallucating, it seems ...
Using showC() from ?complex
showC <- function(z) noquote(sprintf("(R = %g, I = %g)", Re(z), Im(z)))
we see (in R-devel) quite consistently
showC(NA_real_ + 0i)
[1] (R = NA, I = NA)
showC(NA + 0i) # NA is 'logical'
[1] (R = NA, I = NA)
where as in R 4.3.1 and "R-patched" -- *in*consistently
showC(NA_real_ + 0i)
[1] (R = NA, I = 0)
showC(NA + 0i)
[1] (R = NA, I = NA)
and honestly, I do not see *where* (and when) we changed
the underlying code (in arithmetic.c !?) in R-devel to *also*
produce NA_complex_ in such complex *arithmetic*
R_binary() in arithmetic.c has always coerced REALSXP->CPLXSXP when
encountering one of each. Surely then the changes in coerce.c are the
cause and this arithmetic behaviour is just a (bad, IMO) side effect?
> Having said that, one might also (but more naively) expect
> identical(as.complex(as.double(NA_complex_)), NA_complex_)
> to be TRUE.
as in current R-devel
> Under my proposal it continues to be FALSE.
as in "R-release"
> Well, I'd prefer if it gave FALSE with a warning
> "imaginary parts discarded in coercion", but it seems that
> as.double(complex(r=a, i=b)) never warns when either of
> 'a' and 'b' is NA_real_ or NaN, even where "information"
> {nonzero 'b'} is clearly lost ...
The question of *warning* here is related indeed, but I think
we should try to look at it only *secondary* to your first
proposal.
> Whatever decision is made about as.complex(NA_real_),
> maybe these points should be weighed before it becomes part of
> R-release ...
> Mikael
Indeed.
Can we please get other opinions / ideas here?
Thank you, Martin, for "reopening".
Mikael
Thank you in advance for your thoughts!
Martin
---
PS:
Our *print()*ing of complex NA's ("NA" here meaning NA or NaN)
is also unsatisfactory, e.g. in the case where all entries of a
vector are NA in the sense of is.na(.), but their
Re() and Im() are not all NA:
showC <- function(z) noquote(sprintf("(R = %g, I = %g)", Re(z), Im(z)))
z <- complex(, c(11, NA, NA)
[Rd] Recent changes to as.complex(NA_real_)
Revisiting this thread from April:
https://stat.ethz.ch/pipermail/r-devel/2023-April/082545.html
where the decision (not yet backported) was made for as.complex(NA_real_)
to give NA_complex_ instead of complex(r=NA_real_, i=0), to be consistent
with help("as.complex") and as.complex(NA) and as.complex(NA_integer_).
Was any consideration given to the alternative? That is, to changing
as.complex(NA) and as.complex(NA_integer_) to give complex(r=NA_real_, i=0),
consistent with as.complex(NA_real_), then amending help("as.complex")
accordingly?
The principle that Im(as.complex()) should
be zero is quite fundamental, in my view, hence the "new" behaviour seems
to really violate the principle of least surprise ...
Another (but maybe weaker) argument is that double->complex coercions happen
more often than logical->complex and integer->complex ones. Changing the
behaviour of the more frequently performed coercion is more likely to affect
code "out there".
Yet another argument is that one expects
identical(as.complex(NA_real_), NA_real_ + (0+0i))
to be TRUE, i.e., that coercing from double to complex is equivalent to
adding a complex zero. The new behaviour makes the above FALSE, since
NA_real_ + (0+0i) gives complex(r=NA_real_, i=0).
Having said that, one might also (but more naively) expect
identical(as.complex(as.double(NA_complex_)), NA_complex_)
to be TRUE. Under my proposal it continues to be FALSE. Well, I'd prefer
if it gave FALSE with a warning "imaginary parts discarded in coercion",
but it seems that as.double(complex(r=a, i=b)) never warns when either of
'a' and 'b' is NA_real_ or NaN, even where "information" {nonzero 'b'} is
clearly lost ...
Whatever decision is made about as.complex(NA_real_), maybe these points
should be weighed before it becomes part of R-release ...
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R-devel Digest, Vol 246, Issue 6
Two quite related and recent threads on R-devel: [1] proposal for WRE: clarify that use of S4 classes implies use of superclasses https://stat.ethz.ch/pipermail/r-devel/2023-July/082739.html [2] Should package version requirements assume installation from sources? https://stat.ethz.ch/pipermail/r-devel/2022-September/081971.html Notably, if SeuratObject 4.1.3, the current version, had importClassesFrom(Matrix, dgCMatrix, ## and the exported superclasses: CsparseMatrix, Matrix, compMatrix, dMatrix, dsparseMatrix, generalMatrix, sparseMatrix) and not just importClassesFrom(Matrix, dgCMatrix) then the silent breakage due to missing symbols would not have occurred. That is because _imported_ class definitions are retrieved at load time, not at install time. SeuratObject 4.1.3 retrieves the CsparseMatrix definition at install time, so indeed it is liable to become stale with Matrix updates ... Not to derail Dirk's proposal for "Breaks", which could certainly have uses beyond this particular example where a "correct" NAMESPACE directive would solve the problems ... Mikael On 2023-08-07 6:00 am, r-devel-requ...@r-project.org wrote: I would support this suggestion. There is a similar binary dependency chain from Matrix → TMB → glmmTMB; we have implemented various checks to make users aware that they need to reinstall from source, and to some extent we've tried to push out synchronous updates (i.e., push an update of TMB to CRAN every time Matrix changes, and an update of glmmTMB after that), but centralized machinery for this would certainly be nice. FWIW some of the machinery is here: https://github.com/glmmTMB/glmmTMB/blob/d9ee7b043281341429381faa19b5e53cb5a378c3/glmmTMB/R/utils.R#L209-L295 -- it relies on a Makefile rule that caches the current installed version of TMB: https://github.com/glmmTMB/glmmTMB/blob/d9ee7b043281341429381faa19b5e53cb5a378c3/glmmTMB/R/utils.R#L209-L295 cheers Ben Bolker On 2023-08-06 5:05 p.m., Dirk Eddelbuettel wrote: CRAN, by relying on the powerful package management system that is part of R, provides an unparalleled framework for extending R with nearly 20k packages. We recently encountered an issue that highlights a missing element in the otherwise outstanding package management system. So we would like to start a discussion about enhancing its feature set. As shown below, a mechanism to force reinstallation of packages may be needed. A demo is included below, it is reproducible in a container. We find the easiest/fastest reproduction is by saving the code snippet below in the current directory as eg 'matrixIssue.R' and have it run in a container as docker run --rm -ti -v `pwd`:/mnt rocker/r2u Rscript /mnt/matrixIssue.R This runs in under two minutes, first installing the older Matrix, next installs SeuratObject, and then by removing the older Matrix making the (already installed) current Matrix version the default. This simulates a package update for Matrix. Which, as the final snippet demonstrates, silently breaks SeuratObject as the cached S4 method Csparse_validate is now missing. So when SeuratObject was installed under Matrix 1.5.1, it becomes unuseable under Matrix 1.6.0. What this shows is that a call to update.packages() will silently corrupt an existing installation. We understand that this was known and addressed at CRAN by rebuilding all binary packages (for macOS and Windows). But it leaves both users relying on source installation as well as distributors of source packages in a dire situation. It hurt me three times: my default R installation was affected with unit tests (involving SeuratObject) silently failing. It similarly broke our CI setup at work. And it created a fairly bad headache for the Debian packaging I am involved with (and I surmise it affects other distro similarly). It would be good to have a mechanism where a package, when being upgraded, could flag that 'more actions are required' by the system (administrator). We think this example demonstrates that we need such a mechanism to avoid (silently !!) breaking existing installations, possibly by forcing reinstallation of other packages. R knows the package dependency graph and could trigger this, possibly after an 'opt-in' variable the user / admin sets. One possibility may be to add a new (versioned) field 'Breaks:'. Matrix could then have added 'Breaks: SeuratObject (<= 4.1.3)' preventing an installation of Matrix 1.6.0 when SeuratObject 4.1.3 (or earlier) is present, but permitting an update to Matrix 1.6.0 alongside a new version, say, 4.1.4 of SeuratObject which could itself have a versioned Depends: Matrix (>= 1.6.0). Regards, Dirk ## Code example follows. Recommended to run the rocker/r2u container. ## Could also run 'apt update -qq; apt upgrade -y' but not required ## Thanks to my colleague Paul Hoffman for the core of this example ## now have Matrix 1.6.0 because r2u and CRAN
[Rd] proposal for WRE: clarify that use of S4 classes implies use of superclasses
If a package has
importClassesFrom(P, C)
in its NAMESPACE, then should it _also_ have
importClassesFrom(P, )
importClassesFrom(Q, )
## and so on
... ? I think that WRE could be more clear on this point, and in any case
I _think_ that the answer is yes.
Notably, I think that this rule would resolve some of the problems
with stale caches that I described in this thread from September:
https://stat.ethz.ch/pipermail/r-devel/2022-September/081971.html
I've attached a script demonstrating my point. It creates and installs a
package named TestPackage with
importClassesFrom(Matrix, dgCMatrix, CsparseMatrix)
in NAMESPACE and
setClass("whatever", contains = "dgCMatrix")
in R/*.R. After installing the package, I see:
> ns <- asNamespace("TestPackage")
> names(ns)
[1] ".__C__generalMatrix" ".__C__Matrix" ".__C__sparseMatrix"
[4] ".__C__compMatrix" ".__C__whatever" "zzz"
[7] ".packageName" ".__C__dsparseMatrix" ".__NAMESPACE__."
[10] ".__C__dMatrix"".__S3MethodsTable__."
> sort(names(ns))
[1] ".__C__Matrix" ".__C__compMatrix" ".__C__dMatrix"
[4] ".__C__dsparseMatrix" ".__C__generalMatrix" ".__C__sparseMatrix"
[7] ".__C__whatever" ".__NAMESPACE__." ".__S3MethodsTable__."
[10] ".packageName" "zzz"
> sort(names(parent.env(ns)))
[1] ".__C__CsparseMatrix" ".__C__dgCMatrix" "setClass"
i.e., the imported classes are cached in the parent of the namespace (the
imports environment), and superclasses of dgCMatrix that are _not_ imported
are cached in the namespace itself. The caching of superclasses does _not_
occur if I add all of the superclasses to the importClassesFrom directive
in NAMESPACE _or_ if I delete the call to setClass in R/*.R.
It is precisely these cached superclasses that are liable to become stale,
because, if I understand correctly, the namespace is populated and serialized
at _install_ time, whereas the imports environment is populated at _load_ time.
Hence by clarifying in WRE that packages should import exported superclasses
of classes that they extend, etc., we could stem some of the "staleness" on
CRAN ...
But maybe my analysis is mistaken, so I'll wait for comments from others ...
Mikaelpackage <- "TestPackage"
dir.create(file.path(package, "R"), recursive = TRUE)
cat(file = file.path(package, "DESCRIPTION"), "
Package: TestPackage
Version: 0.0-0
License: GPL (>= 2)
Description: A (one paragraph) description of what
the package does and why it may be useful.
Title: My First Collection of Functions
Author: First Last [aut, cre]
Maintainer: First Last
Imports: Matrix, methods
")
cat(file = file.path(package, "NAMESPACE"), "
export(zzz)
importFrom(methods, setClass)
importClassesFrom(Matrix, dgCMatrix, CsparseMatrix)
")
cat(file = file.path(package, "R", paste0(package, ".R")), "
zzz <- function() double(3L)
#setClass(\"whatever\", contains = \"dgCMatrix\")
")
tools:::Rcmd(c("build", package))
tools:::Rcmd(c("INSTALL", Sys.glob(paste0(package, "_*.tar.gz"
## unlink(Sys.glob(paste0(package, "*")), recursive = TRUE)
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] logic tweak needed for as.data.frame. deprecation warning
Another issue raised in the earlier thread is that as.data.frame.POSIXlt
still calls as.data.frame.POSIXct. Hence another path to a false positive
deprecation warning would be:
> Sys.setenv("_R_CHECK_AS_DATA_FRAME_EXPLICIT_METHOD_" = TRUE)
> as.data.frame(as.POSIXlt(.POSIXct(0, "UTC")))
as.POSIXlt(.POSIXct(0, "UTC"))
1 1970-01-01
Warning message:
Direct call of 'as.data.frame.POSIXct()' is deprecated. Use
'as.data.frame.vector()' or 'as.data.frame()' instead
as.data.frame.POSIXlt could simply change to using as.data.frame.vector.
I glanced at the other non-deprecated as.data.frame. and did not
see other usage of the deprecated ones ... a more systematic check could
be worth doing.
Mikael
On 2023-07-06 11:32 am, Mikael Jagan wrote:
Continuing the thread started on R-package-devel, here:
https://stat.ethz.ch/pipermail/r-package-devel/2023q3/009307.html
The logic of the now soft-deprecated as.data.frame.,
> body(as.data.frame.integer)[[2L]]
if ((sys.nframe() <= 1L || sys.call(-1L)[[1L]] != quote(as.data.frame)) &&
nzchar(Sys.getenv("_R_CHECK_AS_DATA_FRAME_EXPLICIT_METHOD_")))
.Deprecated(msg = gettextf("Direct call of '%s()' is deprecated. Use '%s()' or
'%s()' instead",
"as.data.frame.integer", "as.data.frame.vector", "as.data.frame"))
may need adjustment to avoid false positives such as this one:
> Sys.setenv("_R_CHECK_AS_DATA_FRAME_EXPLICIT_METHOD_" = TRUE)
> f <- as.data.frame
> f(0L)
0L
1 0
Warning message:
Direct call of 'as.data.frame.integer()' is deprecated. Use
'as.data.frame.vector()' or 'as.data.frame()' instead
i.e., the condition sys.call(-1L)[[1L]] != quote(as.data.frame) is not precise
enough ... would !identical(sys.function(-1L), as.data.frame) work instead?
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] logic tweak needed for as.data.frame. deprecation warning
Continuing the thread started on R-package-devel, here:
https://stat.ethz.ch/pipermail/r-package-devel/2023q3/009307.html
The logic of the now soft-deprecated as.data.frame.,
> body(as.data.frame.integer)[[2L]]
if ((sys.nframe() <= 1L || sys.call(-1L)[[1L]] != quote(as.data.frame)) &&
nzchar(Sys.getenv("_R_CHECK_AS_DATA_FRAME_EXPLICIT_METHOD_")))
.Deprecated(msg = gettextf("Direct call of '%s()' is deprecated. Use '%s()' or
'%s()' instead",
"as.data.frame.integer", "as.data.frame.vector", "as.data.frame"))
may need adjustment to avoid false positives such as this one:
> Sys.setenv("_R_CHECK_AS_DATA_FRAME_EXPLICIT_METHOD_" = TRUE)
> f <- as.data.frame
> f(0L)
0L
1 0
Warning message:
Direct call of 'as.data.frame.integer()' is deprecated. Use
'as.data.frame.vector()' or 'as.data.frame()' instead
i.e., the condition sys.call(-1L)[[1L]] != quote(as.data.frame) is not precise
enough ... would !identical(sys.function(-1L), as.data.frame) work instead?
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] New behavior when running script in package directory?
Surely this behaviour is just a case of ESS being "too clever", sourcing
*.R files in special way when it detects that a file belongs to a package
(loading dependencies automatically, etc.)?
The function ss() is defined inside of .ess.source(), which is defined here:
https://github.com/emacs-ess/ESS/blob/5c4ae91cefa5c56fd13b204a9a996825af836a67/etc/ESSR/R/.basic.R#L168
If you think that there is a bug, then you could report it there ...
Mikael
On 2023-06-21 6:00 am, r-devel-requ...@r-project.org wrote:
When I run a script foo.R containing some trivial code in my home
directory, via Emacs/ESS, everything works as expected: R
starts, and a setwd() command to set the working directory is
run automatically before the code in the script is run.
But if I copy foo.R to some package/R directory strange
things happen. When I use Emacs/ESS to run the script
in its new location, R starts, and setwd() is called to set
the working directory, but then one or more libraries that the
package depends on are loaded, even though I am using no
libraries in foo.R.
Now consider foo.R that contains the following trivial code:
secsToRDateTime <- function(secs) {
day2sec <- 60*60*24
days <- secs/day2sec
}
When I try to run this from package/R I get...
Error in ss(file, echo = visibly, local = local, print.eval = output, :
/tmp/gpstime.R!CuSewT:2:0: unexpected end of input
1: secsToRDateTime <- function(secs) {
^
As I said, there are no problems when the script is run from my
home directory. This suggests that test scripts can no longer be
tested in a package's R directory?
Is this true?
Thanks,
Dominick
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] codetools wrongly complains about lazy evaluation in S4 methods
On 2023-06-15 5:25 pm, Hervé Pagès wrote:
Oh but I see now that you've already tried this in your R/AllGenerics.R,
sorry for missing that, but that you worry about the following message
being disruptive on CRAN:
The following object is masked from 'package:base':
qr.X
Why would that be? As long as you only define methods for objects that
**you** control everything is fine. In other words you're not allowed to
define a method for "qr" objects because that method would override
base::qr.X(). But the generic itself and the method that you define for
your objects don't override anything so should not disrupt anything.
Yes, maybe it would be fine in principle, and of course many popular
packages emit startup messages. Still, in practice, I think that
people are quite accustomed to library(Matrix) being "silent",
and probably a nontrivial fraction of our reverse dependencies would
encounter new NOTEs about differences between *.Rout and *.Rout.save,
etc.
The fraction of users who will ever call this method for qr.X is very
very small compared to the fraction who will be confused or annoyed
by the message. Hence my hope that an implicit generic qr.X could
become part of package methods, notably as an implicit generic qr.R
already lives there ...
Or maybe there is a way for Matrix to define qr.X as an implicit generic
without creating other problems, but my experiments with setGenericImplicit
were not promising ...
Mikael
H.
On 6/15/23 13:51, Hervé Pagès wrote:
I'd argue that at the root of the problem is that your qr.X() generic
dispatches on all its arguments, including the 'ncol' argument which I
think the dispatch mechanism needs to evaluate **before** dispatch can
actually happen.
So yes lazy evaluation is a real feature but it does not play well for
arguments of a generic that are involved in the dispatch.
If you explicitly defined your generic with:
setGeneric("qr.X", signature="qr")
you should be fine.
More generally speaking, it's a good idea to restrict the signature of
a generic to the arguments "that make sense". For unary operations
this is usually the 1st argument, for binary operations the first two
arguments etc... Additional arguments that control the operation like
modiflers, toggles, flags, rng seed, and other parameters, usually
have not place in the signature of the generic.
H.
On 6/14/23 20:57, Mikael Jagan wrote:
Thanks all - yes, I think that Simon's diagnosis ("user error") is
correct:
in this situation one should define a reasonable generic function
explicitly,
with a call to setGeneric, and not rely on the call inside of
setMethod ...
But it is still not clear what the way forward should be (for package
Matrix,
where we would like to export a method for 'qr.X'). If we do
nothing, then
there is the note, already mentioned:
* checking R code for possible problems ... NOTE
qr.X: no visible binding for global variable ‘R’
Undefined global functions or variables:
R
If we add the following to our R/AllGenerics.R :
setGeneric("qr.X",
function(qr, complete = FALSE, ncol, ...)
standardGeneric("qr.X"),
useAsDefault = function(qr, complete = FALSE, ncol,
...) {
if(missing(ncol))
base::qr.X(qr, complete = complete)
else base::qr.X(qr, complete = complete, ncol = ncol)
},
signature = "qr")
then we get a startup message, which would be quite disruptive on CRAN :
The following object is masked from 'package:base':
qr.X
and if we further add setGenericImplicit("qr.X", restore = (TRUE|FALSE))
to our R/zzz.R, then for either value of 'restore' we encounter :
** testing if installed package can be loaded from temporary
location
Error: package or namespace load failed for 'Matrix':
Function found when exporting methods from the namespace
'Matrix' which is not S4 generic: 'qr.X'
Are there possibilities that I have missed?
It seems to me that the best option might be to define an implicit
generic
'qr.X' in methods via '.initImplicitGenerics' in
methods/R/makeBasicFunsList.R,
where I see that an implicit generic 'qr.R' is already defined ... ?
The patch pasted below "solves everything", though we'd still have to
think
about how to work for versions of R without the patch ...
Mikael
Index: src/library/methods/R/makeBasicFunsList.R
===
--- src/library/methods/R/makeBasicFunsList.R (revision 84541)
+++ src/library/methods/R/makeBasicFunsList.R (working copy)
@@ -263,6 +263,17 @@
signature = "qr", where = where)
setGenericImplicit("qr.R", where, FALSE)
+ setGeneric("qr.X",
+ function(qr,
Re: [Rd] codetools wrongly complains about lazy evaluation in S4 methods
Thanks all - yes, I think that Simon's diagnosis ("user error") is correct:
in this situation one should define a reasonable generic function explicitly,
with a call to setGeneric, and not rely on the call inside of setMethod ...
But it is still not clear what the way forward should be (for package Matrix,
where we would like to export a method for 'qr.X'). If we do nothing, then
there is the note, already mentioned:
* checking R code for possible problems ... NOTE
qr.X: no visible binding for global variable ‘R’
Undefined global functions or variables:
R
If we add the following to our R/AllGenerics.R :
setGeneric("qr.X",
function(qr, complete = FALSE, ncol, ...)
standardGeneric("qr.X"),
useAsDefault = function(qr, complete = FALSE, ncol, ...) {
if(missing(ncol))
base::qr.X(qr, complete = complete)
else base::qr.X(qr, complete = complete, ncol = ncol)
},
signature = "qr")
then we get a startup message, which would be quite disruptive on CRAN :
The following object is masked from 'package:base':
qr.X
and if we further add setGenericImplicit("qr.X", restore = (TRUE|FALSE))
to our R/zzz.R, then for either value of 'restore' we encounter :
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for 'Matrix':
Function found when exporting methods from the namespace 'Matrix' which is
not S4 generic: 'qr.X'
Are there possibilities that I have missed?
It seems to me that the best option might be to define an implicit generic
'qr.X' in methods via '.initImplicitGenerics' in methods/R/makeBasicFunsList.R,
where I see that an implicit generic 'qr.R' is already defined ... ?
The patch pasted below "solves everything", though we'd still have to think
about how to work for versions of R without the patch ...
Mikael
Index: src/library/methods/R/makeBasicFunsList.R
===
--- src/library/methods/R/makeBasicFunsList.R (revision 84541)
+++ src/library/methods/R/makeBasicFunsList.R (working copy)
@@ -263,6 +263,17 @@
signature = "qr", where = where)
setGenericImplicit("qr.R", where, FALSE)
+setGeneric("qr.X",
+ function(qr, complete = FALSE, ncol, ...)
+ standardGeneric("qr.X"),
+ useAsDefault = function(qr, complete = FALSE, ncol, ...) {
+ if(missing(ncol))
+ base::qr.X(qr, complete = complete)
+ else base::qr.X(qr, complete = complete, ncol = ncol)
+ },
+ signature = "qr", where = where)
+setGenericImplicit("qr.X", where, FALSE)
+
## our toeplitz() only has 'x'; want the generic "here" rather than "out
there"
setGeneric("toeplitz", function(x, ...) standardGeneric("toeplitz"),
useAsDefault= function(x, ...) stats::toeplitz(x),
On 2023-06-13 8:01 pm, Simon Urbanek wrote:
I agree that this is not an R issue, but rather user error of not defining a
proper generic so the check is right. Obviously, defining a generic with
implementation-specific ncol default makes no sense at all, it should only be
part of the method implementation. If one was to implement the same default
behavior in the generic itself (not necessarily a good idea) the default would
be ncol = if (complete) nrow(qr.R(qr, TRUE)) else min(dim(qr.R(qr, TRUE))) to
not rely on the internals of the implementation.
Cheers,
Simon
On 14/06/2023, at 6:03 AM, Kasper Daniel Hansen
wrote:
On Sat, Jun 3, 2023 at 11:51 AM Mikael Jagan wrote:
The formals of the newly generic 'qr.X' are inherited from the non-generic
function in the base namespace. Notably, the inherited default value of
formal argument 'ncol' relies on lazy evaluation:
formals(qr.X)[["ncol"]]
if (complete) nrow(R) else min(dim(R))
where 'R' must be defined in the body of any method that might evaluate
'ncol'.
Perhaps I am misunderstanding something, but I think Mikael's expectations
about the scoping rules of R are wrong. The enclosing environment of ncol
is where it was _defined_ not where it is _called_ (apologies if I am
messing up the computer science terminology here).
This suggests to me that codetools is right. But a more extended example
would be useful. Perhaps there is something special with setOldClass()
which I am no aware of.
Also, Bioconductor has 100s of packages with S4 where codetools works well.
Kasper
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] codetools wrongly complains about lazy evaluation in S4 methods
Thanks both. Yes, I was aware of globalVariables, etc. I guess I was hoping
to be pointed to the right place in the source code, in case the issue could
be addressed properly, notably as it seems to have already been addressed for
functions that are not S4 methods, i.e., codetools is apparently not bothered
by
def <- function(x = y) { y <- 0; x }
but still complains about
setMethod("someGeneric", "someClass", def)
...
Mikael
On 2023-06-07 5:13 am, Gabriel Becker wrote:
The API supported workaround is to call globalVariables, which,
essentially, declares the variables without defining them (a distinction R
does not usually make).
The issue with this approach, of course, is that its a very blunt
instrument. It will cause false negatives if you accidentally use the same
symbol in a standard evaluation context elsewhere in your code.
Nonetheless, that's the intended approach as far as i know.
Best,
~G
On Wed, Jun 7, 2023 at 1:07 AM Serguei Sokol via R-devel <
r-devel@r-project.org> wrote:
Le 03/06/2023 à 17:50, Mikael Jagan a écrit :
In a package, I define a method for not-yet-generic function 'qr.X'
like so:
> setOldClass("qr")
> setMethod("qr.X", signature(qr = "qr"), function(qr, complete,
ncol) NULL)
The formals of the newly generic 'qr.X' are inherited from the
non-generic
function in the base namespace. Notably, the inherited default value of
formal argument 'ncol' relies on lazy evaluation:
> formals(qr.X)[["ncol"]]
if (complete) nrow(R) else min(dim(R))
where 'R' must be defined in the body of any method that might
evaluate 'ncol'.
To my surprise, tools:::.check_code_usage_in_package() complains about
the
undefined symbol:
qr.X: no visible binding for global variable 'R'
qr.X,qr: no visible binding for global variable 'R'
Undefined global functions or variables:
R
I think this issue is similar to the complaints about non defined
variables in expressions involving non standard evaluation, e.g. column
names in a data frame which are used as unquoted symbols. One of
workarounds is simply to declare them somewhere in your code. In your
case, it could be something as simple as:
R=NULL
Best,
Serguei.
I claim that it should _not_ complain, given that lazy evaluation is a
really
a feature of the language _and_ given that it already does not
complain about
the formals of functions that are not S4 methods.
Having said that, it is not obvious to me what in codetools would need
to change
here. Any ideas?
I've attached a script that creates and installs a test package and
reproduces
the check output by calling tools:::.check_code_usage_in_package().
Hope it
gets through.
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Rd macros are not expanded inside of \eqn{} or \deqn{}
I was a bit surprised to learn that, if one has an Rd file as below: %% zzz.Rd \newcommand{\zzz}{whatever} \name{zzz} \title{zzz} \description{ \zzz{} \eqn{\zzz{}} \deqn{\zzz{}} } then the macro is _not_ expanded inside of \eqn{} or \deqn{} when parsed to text or HTML. Is this behaviour intentional? Could it be changed? Inside of \eqn{} and \deqn{} is where I am _most_ likely to want to use macros, at least since R 4.2.0, which added KaTeX support ... See output pasted below. Mikael > tools::Rd2txt(tools::parse_Rd("zzz.Rd")) zzz Description: whatever \zzz{} \zzz{} > tools::Rd2HTML(tools::parse_Rd("zzz.Rd")) R: zzz href="https://cdn.jsdelivr.net/npm/katex@0.15.3/dist/katex.min.css;> const macros = { "\\R": "\\textsf{R}", "\\code": "\\texttt"}; function processMathHTML() { var l = document.getElementsByClassName('reqn');for (let e of l) { katex.render(e.textContent, e, { throwOnError: false, macros }); }return; } https://cdn.jsdelivr.net/npm/katex@0.15.3/dist/katex.min.js"; onload="processMathHTML();"> zzzR Documentation zzz Description whatever \zzz{} \zzz{} __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] codetools wrongly complains about lazy evaluation in S4 methods
In a package, I define a method for not-yet-generic function 'qr.X' like so:
> setOldClass("qr")
> setMethod("qr.X", signature(qr = "qr"), function(qr, complete, ncol) NULL)
The formals of the newly generic 'qr.X' are inherited from the non-generic
function in the base namespace. Notably, the inherited default value of
formal argument 'ncol' relies on lazy evaluation:
> formals(qr.X)[["ncol"]]
if (complete) nrow(R) else min(dim(R))
where 'R' must be defined in the body of any method that might evaluate 'ncol'.
To my surprise, tools:::.check_code_usage_in_package() complains about the
undefined symbol:
qr.X: no visible binding for global variable 'R'
qr.X,qr: no visible binding for global variable 'R'
Undefined global functions or variables:
R
I claim that it should _not_ complain, given that lazy evaluation is a really
a feature of the language _and_ given that it already does not complain about
the formals of functions that are not S4 methods.
Having said that, it is not obvious to me what in codetools would need to change
here. Any ideas?
I've attached a script that creates and installs a test package and reproduces
the check output by calling tools:::.check_code_usage_in_package(). Hope it
gets through.
Mikaelnm <- "TestPackage"
dir.create(tmp <- file.path(tempdir(), nm))
dir.create(file.path(tmp, "R"))
cat(file = file.path(tmp, "DESCRIPTION"), "
Package: TestPackage
Version: 0.0-0
License: GPL (>= 2)
Description: A (one paragraph) description of what
the package does and why it may be useful.
Title: My First Collection of Functions
Author: First Last [aut, cre]
Maintainer: First Last
Imports: methods
")
cat(file = file.path(tmp, "NAMESPACE"), "
importFrom(methods, setMethod, setOldClass)
exportMethods(qr.X)
export(xyz)
")
cat(file = file.path(tmp, "R", paste0(nm, ".R")), "
setOldClass(\"qr\")
setMethod(\"qr.X\", signature(qr = \"qr\"), function(qr, complete, ncol) NULL)
xyz <- function(x, y = z) { z <- 1; x + y }
")
install.packages(tmp, repos = NULL)
tools:::.check_code_usage_in_package(nm)
## qr.X: no visible binding for global variable 'R'
## qr.X,qr: no visible binding for global variable 'R'
## Undefined global functions or variables:
## R
unloadNamespace(nm)
remove.packages(nm)
unlink(tmp)
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Change DEFAULTDEPARSE to DEFAULTDEPARSE | SHOWATTRIBUTES ?
The deparse options used by default by 'deparse' and 'dput' are
c("keepNA", "keepInteger", "niceNames", "showAttributes")
but Defn.h still has
#define DEFAULTDEPARSE 1089 /* KEEPINTEGER | KEEPNA | NICE_NAMES, used for
calls */
i.e., with the SHOWATTRIBUTES bit turned off. Is that on purpose?
Note that this leads to weird things like:
> (expr <- call("is.matrix", matrix(1:4, 2L, 2L)))
is.matrix(1:4)
> eval(expr)
[1] TRUE
which can confuse anyone not paying close attention ...
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] sum(), min(), max(), prod() vs. named arguments in ...
On 2023-04-15 6:00 am, r-devel-requ...@r-project.org wrote:
Date: Fri, 14 Apr 2023 13:38:00 +0300
From: Ivan Krylov
To:r-devel@r-project.org
Subject: [Rd] sum(), min(), max(), prod() vs. named arguments in ...
Message-ID: <20230414133800.75383dae-6792@arachnoid>
Content-Type: text/plain; charset="us-ascii"
Hello R-devel,
As mentioned on Fosstodon [1] and discussed during RCOH [2], named
arguments to sum(), min(), max() and prod() are currently processed the
same way as the un-named ones. Additionally, when specifying the na.rm
argument more than once, the last specified value is silently taken
into account.
Most of this behaviour is exactly as documented, but it's arguably a
source of mistakes when the na.rm=TRUE argument is mistyped and
converted to 1.
Internally, all these calls land in do_summary(), and the na.rm
argument is processed by fixup_NaRm(). mean() is safe because it's
dispatched via S3 lookup.
Do users normally name their arguments to sum()? If not, it could be
relatively safe to make it a warning to pass named arguments to sum()
that are not na.rm and later transition it to an error:
Forbidding or warning against tagged arguments other than 'na.rm' would be
quite intrusive, since it is not uncommon to see do.call(sum, ).
--- src/main/summary.c (revision 84252)
+++ src/main/summary.c (working copy)
@@ -419,6 +419,8 @@
na_value = CAR(a);
if(prev == R_NilValue) args = CDR(a);
else SETCDR(prev, CDR(a));
+ } else if (TAG(a) && TAG(a) != R_NilValue) {
+ warning("Named argument \"%s\" here is probably a mistake",
CHAR(PRINTNAME(TAG(a;
}
prev = a;
}
(Can TAG(a) ever be NULL or anything other than R_NilValue or a symbol
here? We'll probably need to translate this message, too.)
Passing na.rm more than once could be made into an error right away:
That would be much easier to support, as users should be accustomed to that
_not_ working.
--- src/main/summary.c (revision 84252)
+++ src/main/summary.c (working copy)
@@ -409,6 +409,7 @@
attribute_hidden
SEXP fixup_NaRm(SEXP args)
{
+Rboolean seen_NaRm = FALSE;
SEXP t, na_value;
/* Need to make sure na.rm is last and exists */
@@ -415,7 +416,9 @@
na_value = ScalarLogical(FALSE);
for(SEXP a = args, prev = R_NilValue; a != R_NilValue; a = CDR(a)) {
if(TAG(a) == R_NaRmSymbol) {
+ if(seen_NaRm) error("Please specify na.rm only once");
+ seen_NaRm = TRUE;
if(CDR(a) == R_NilValue) return args;
na_value = CAR(a);
if(prev == R_NilValue) args = CDR(a);
else SETCDR(prev, CDR(a));
(Can we use error(_("formal argument \"%s\" matched by multiple actual
arguments"), "na.rm") to emit an already translated error message?)
Additionally, is it desirable to have S3 methods for sum() and its
friends? Currently, they are dispatched, but something confusing
happens to the arguments: according to the method, the dots contain
what I would expect them to contain (only the numbers to sum), but when
calling NextMethod() to compute the actual sum, extra arguments get
processed:
sum.foo <- function(..., ExtraArg = 999, na.rm = FALSE) {
print(ExtraArg)
str(...)
NextMethod(generic = 'sum', ..., na.rm = na.rm)
}
sum(structure(100, class = 'foo'))
# [1] 999
# 'foo' num 100
# [1] 100
sum(structure(100, class = 'foo'), ExtraArg = -1)
# [1] -1
# 'foo' num 100 # <-- ExtraArg = -1 missing from ...
# [1] 99 # <-- but still subtracted
str(...) only gives the structure of the first argument matching the dots.
'ExtraArg' doesn't match the dots, as you say, but even if it did, it would
be discarded silently by 'str' there. Better would be, e.g., str(list(...)).
Maybe I'm just misusing NextMethod(). This is probably not important in
practice due to how S3 dispatch works.
In both of your examples, just one named "actual" matches the "formal" '...'
in the call to 'NextMethod': the promise 'na.rm' evaluating to FALSE.
Named actuals matching the formal '...' are used to update the original call:
sum(structure(100, class = 'foo'))
==> sum(structure(100, class = 'foo'), na.rm = na.rm)
sum(structure(100, class = 'foo'), ExtraArg = -1)
==> sum(structure(100, class = 'foo'), ExtraArg = -1, na.rm = na.rm)
The new call is evaluated, this time dispatching the internal default method,
giving the results that you observed.
FWIW, help("NextMethod") does warn:
When a primitive is called as the default method, argument matching may
not work as described above due to the different semantics of primitives.
Mikael
-- Best regards, Ivan [1] https://fosstodon.org/@nibsalot/110105034507818285 [2]
https://github.com/r-devel/rcontribution/blob/main/office_hours/2023-04-13_EMEA-APAC.md
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Time to add is.formula() to 'stats'?
I know that it has been discussed in the past, but I wanted to ask
to revisit the idea of exporting
is.formula <- function(x) inherits(x, "formula")
from 'stats', parallel to is.data.frame() in 'base', given how
widely formulae are used these days in conjunction with data frames,
even outside of model fitting functions (e.g., for split-apply).
One could argue that today data frames and formulae go hand in hand,
and that the lack of is.formula() is a rather ugly asymmetry ...
Furthermore, 'formula' is one of the only S3 classes generated by way
of a primitive (the `~` operator, calling do_tilde() in names.c), so
it is really in some sense "special" { compared to 'factor', 'POSIXlt',
etc. }.
What do people think?
In case it helps, I've gathered some data from 'base' and the so-called
'defaultPackages', reproduced by the attached R script ... see below.
Mikael
1. For which X does is.X() have no corresponding as.X()?
[1] "R" "atomic""element" "finite""hashtab" "infinite"
[7] "language" "leaf" "loaded""mts" "na""nan"
[13] "object""primitive" "recursive" "tskernel" "unsorted"
2. For which Y does as.Y() have no corresponding is.Y()?
[1] "Date" "POSIXct" "POSIXlt" "dendrogram"
[5] "difftime" "dist" "formula" "graphicsAnnot"
[9] "hclust""hexmode" "octmode" "person"
[13] "personList""roman"
3. For which Z does is.Z() just call inherits(., "Z")?
[1] "data.frame" "factor" "numeric_version" "ordered"
[5] "package_version" "raster" "relistable" "table"
## all as.*() and is.*() in 'base' and 'defaultPackages'
pkg <- c("base", "datasets", "grDevices", "graphics",
"methods", "stats", "utils")
getExports <- function(s) names(as.environment(paste0("package:", s)))
nms <- sort(unlist(lapply(pkg, getExports)))
as. <- nms[startsWith(nms, "as.")]
is. <- nms[startsWith(nms, "is.")]
## excluding methods and assignments
as.. <- c("as.data.frame", grep("([.].+[.]|<-)", as., value=TRUE, invert=TRUE))
is.. <- c("is.data.frame", grep("([.].+[.]|<-)", is., value=TRUE, invert=TRUE))
cl.as <- sub("^as[.]", "", as..)
cl.is <- sub("^is[.]", "", is..)
callsInherits <- function(x)
is.call(b <- body(match.fun(x))) && b[[1L]] == "inherits"
## missing as.*()
setdiff(cl.is, cl.as)
## missing is.*()
setdiff(cl.as, cl.is)
## is.*() just calling inherits()
cl.is[vapply(is.., callsInherits, NA)]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] det(diag(c(NaN, 1))) should be NaN, not 0
Hmm ... I can no longer reproduce this under r83996 when configuring
--without-lapack and --without-blas {the default}. Now det(A), det(B),
det(C), and det(D) are all NaN.
I assume that the reason is the recent update to use LAPACK 3.11.0?
But I don't see any related NEWS here:
https://netlib.org/lapack/lapack-3.11.0.html
It is possible that the underlying issue in DGETRF (INFO>0 due to NaN
pivots rather than 0 pivots {=> singular}) still exists for matrices
less trivial than my 2-by-2 examples. Will have to experiment ...
Mikael
On 2022-11-09 3:58 pm, Mikael Jagan wrote:
Hello,
Currently, determinant(A) calculates the determinant of 'A' by factorizing
A=LU and computing prod(diag(U)) [or the logarithm of the absolute value].
The factorization is done by LAPACK routine DGETRF, which gives a status
code INFO, documented [1] as follows:
*> INFO is INTEGER
*> = 0: successful exit
*> < 0: if INFO = -i, the i-th argument had an illegal value
*> > 0: if INFO = i, U(i,i) is exactly zero. The factorization
*>has been completed, but the factor U is exactly
*>singular, and division by zero will occur if it is used
*>to solve a system of equations.
Accordingly, when INF0>0, determinant(A) behaves as det(A)=0, _not_ computing
prod(diag(U)). The problem here is that DGETRF can _also_ give positive
INFO for matrices containing NaN, which may very well not be singular for some
finite value of NaN.
I claim that, when INFO>0, determinant(A) should _not_ behave as det(A)=0
unconditionally, but rather sometimes (depending on some test) give NaN.
Here is one case where 0 is really "wrong":
> (A <- diag(c(NaN, 1)))
[,1] [,2]
[1,] NaN0
[2,]01
> det(A)
[1] 0
R isn't consistent, either:
> (B <- diag(c(1, NaN)))
[,1] [,2]
[1,]10
[2,]0 NaN
> det(B)
[1] NaN
Here, DGETRF _does_ succeed, because it does not "see" the trailing NaN in 'B'.
So: Should R change to better handle the INFO>0 case? If so, how?
Ideally (I think), the proposed change would give NaN for 'A' and 'B'
above and 0 for 'C' and 'D' below (both of which really _are_ singular):
> (C <- matrix(c(NaN, NaN, 0, 0), 2L, 2L))
[,1] [,2]
[1,] NaN0
[2,] NaN0
> det(C)
[1] NaN
> (D <- t(C))
[,1] [,2]
[1,] NaN NaN
[2,]00
> det(D)
[1] 0
Furthermore, the proposed change should _not_ decrease the performance
of determinant(A) for nonsingular 'A' ...
For those looking, the relevant C-level function is det_ge_real(),
defined in R-devel/src/modules/lapack/Lapack.c (at line 1260 in r83320).
Mikael
[1] https://github.com/Reference-LAPACK/lapack/blob/master/SRC/dgetrf.f
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Regenerate m4/gettext.m4 with updated AM_GNU_GETTEXT?
Currently, m4/gettext.m4 contains an old version of Autoconf macro AM_GNU_GETTEXT relying on internal symbols (e.g., _nl_expand_alias) to detect broken implementations of GNU gettext. System libraries are searched for the internal symbols unconditionally, i.e., even when linking against a static GNU libintl, resulting in the following false positive in my config.log: configure:53900: checking for GNU gettext in libintl configure:53937: clang -o conftest -g -O2 -Wall -pedantic -mmacosx-version-min=11.0 -arch arm64 -falign-functions=8 -Wno-error=implicit-function-declaration -fPIC -DPCRE2_STATIC -I/usr/local/include -I/opt/R/arm64/include -I/usr/local/include -L/opt/R/arm64/lib -L/usr/local/lib conftest.c -L/usr/local/lib -lpcre2-8 -llzma -lbz2 -lz -licucore -ldl -lm -liconv /usr/local/lib/libintl.a /usr/local/lib/libiconv.a >&5 Undefined symbols for architecture arm64: "__nl_expand_alias", referenced from: _main in conftest-0e23fb.o ld: symbol(s) not found for architecture arm64 AM_GNU_GETTEXT was revised in the GNU gettext sources ~6 years ago to address this: $ git log -1 b67399b commit b67399b40bc5bf3165b09e6a095ec941d4b30a97 Author: Daiki Ueno Date: Thu May 26 13:38:57 2016 +0900 m4: Rely less on internal symbols * gettext-runtime/m4/gettext.m4 (AM_GNU_GETTEXT): Skip checks for the internal symbols _nl_msg_cat_cntr, _nl_domain_bindings, and _nl_expand_alias, if __GNU_GETTEXT_SUPPORTED_REVISION is defined. Problem reported and fix suggested by Masanori Ogino in: https://lists.gnu.org/archive/html/bug-gettext/2016-04/msg0.html The rationale behind this is: (1) those symbol checks are for detecting certain broken implementations, namely NetBSD and Solaris 7, and (2) __GNU_GETTEXT_SUPPORTED_REVISION is not defined in those implementations, while it is defined on compatible implementations, such as musl-libc which doesn't have those internal symbols. Can m4/gettext.m4 be regenerated from a sufficiently newer release of GNU gettext, without major disruption? Mikael __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] solve.default(a, b) should not ignore names(dimnames(.))
On 2022-11-10 1:10 pm, Mikael Jagan wrote:
Hello,
Currently, solve.default() is defined such that
> identical(dimnames(solve.default(a, b)),
list(dimnames(a)[[2L]], dimnames(b)[[2L]]))
is always TRUE, i.e., ignoring names(dimnames(a)) and names(dimnames(b)).
Hence we see:
> a <- b <- diag(2L)
> dimnames(a) <- list(A1 = c("a11", "a12"), A2 = c("a21", "a22"))
> dimnames(b) <- list(B1 = c("b11", "b12"), B2 = c("b21", "b22"))
> a
A2
A1a21 a22
a11 1 0
a12 0 1
> b
B2
B1b21 b22
b11 1 0
b12 0 1
> solve.default(a, b)
b21 b22
a21 1 0
a22 0 1
I claim that solve.default() should be changed to instead give:
> identical(dimnames(solve.default(a, b)),
c(dimnames(a)[2L], dimnames(b)[2L]))
This would make solve.default() consistent with `%*%`, which
_does_ respect names(dimnames(.)) :
> a %*% b
B2
A1b21 b22
a11 1 0
a12 0 1
If others agree, then I would submit a minimal patch to the R-level
solve.default() in src/library/base/R/solve.R and to the C-level
La_solve() and La_solve_cmplx() in src/modules/lapack/Lapack.c ...
Well, taking a closer look, I see more related issues, which might
be considered simultaneously. (Sorry if this is too much for one thread.)
* qr.solve(a, b) and solve.qr(qr(a), b) also ignore names(dimnames(.))
> qr.solve(a, b)
b21 b22
a21 1 0
a22 0 1
> solve.qr(qr(a), b)
b21 b22
a21 1 0
a22 0 1
* dimnames(qr.solve(a)) and dimnames(solve.qr(qr(a))) do not agree
with dimnames(solve.default(a)), i.e., when 'b' is missing
> solve.default(a)
a11 a12
a21 1 0
a22 0 1
> qr.solve(a)
[,1] [,2]
a2110
a2201
> solve.qr(qr(a))
[,1] [,2]
a2110
a2201
* More controversially: qr.qy(qr, y) and qr.qty(qr, y) are currently
documented as retaining the 'dimnames' of 'y', and they _do_. But
_conventionally_ the rownames of matrix products are taken from the
first factor, in this case 'Q'. It is not currently documented what
the 'dimnames' of the implicitly stored 'Q' and 'R' factors should be,
leading to inconsistencies like this:
> a
A2
A1a21 a22
a11 1 0
a12 0 1
> qr.Q(qr(a)) %*% qr.R(qr(a))
A2
a21 a22
[1,] 1 0
[2,] 0 1
Hence I propose to enforce {and partly document} the following:
1. that Q=qr.Q(qr(X)) gets the rownames of X and NULL colnames
2. that R=qr.R(qr(X)) gets the (permuted) colnames of X and NULL rownames
3. that dimnames(qr.qy (qr(X), y)) <=> dimnames( Q %*% y)
4. that dimnames(qr.qty(qr(X), y)) <=> dimnames(t(Q) %*% y)
I'm not sure what the level of disruption on CRAN would be,
but internal consistency here would certainly be nice ...
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] solve.default(a, b) should not ignore names(dimnames(.))
Hello,
Currently, solve.default() is defined such that
> identical(dimnames(solve.default(a, b)),
list(dimnames(a)[[2L]], dimnames(b)[[2L]]))
is always TRUE, i.e., ignoring names(dimnames(a)) and names(dimnames(b)).
Hence we see:
> a <- b <- diag(2L)
> dimnames(a) <- list(A1 = c("a11", "a12"), A2 = c("a21", "a22"))
> dimnames(b) <- list(B1 = c("b11", "b12"), B2 = c("b21", "b22"))
> a
A2
A1a21 a22
a11 1 0
a12 0 1
> b
B2
B1b21 b22
b11 1 0
b12 0 1
> solve.default(a, b)
b21 b22
a21 1 0
a22 0 1
I claim that solve.default() should be changed to instead give:
> identical(dimnames(solve.default(a, b)),
c(dimnames(a)[2L], dimnames(b)[2L]))
This would make solve.default() consistent with `%*%`, which
_does_ respect names(dimnames(.)) :
> a %*% b
B2
A1b21 b22
a11 1 0
a12 0 1
If others agree, then I would submit a minimal patch to the R-level
solve.default() in src/library/base/R/solve.R and to the C-level
La_solve() and La_solve_cmplx() in src/modules/lapack/Lapack.c ...
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] det(diag(c(NaN, 1))) should be NaN, not 0
Hello, Currently, determinant(A) calculates the determinant of 'A' by factorizing A=LU and computing prod(diag(U)) [or the logarithm of the absolute value]. The factorization is done by LAPACK routine DGETRF, which gives a status code INFO, documented [1] as follows: *> INFO is INTEGER *> = 0: successful exit *> < 0: if INFO = -i, the i-th argument had an illegal value *> > 0: if INFO = i, U(i,i) is exactly zero. The factorization *>has been completed, but the factor U is exactly *>singular, and division by zero will occur if it is used *>to solve a system of equations. Accordingly, when INF0>0, determinant(A) behaves as det(A)=0, _not_ computing prod(diag(U)). The problem here is that DGETRF can _also_ give positive INFO for matrices containing NaN, which may very well not be singular for some finite value of NaN. I claim that, when INFO>0, determinant(A) should _not_ behave as det(A)=0 unconditionally, but rather sometimes (depending on some test) give NaN. Here is one case where 0 is really "wrong": > (A <- diag(c(NaN, 1))) [,1] [,2] [1,] NaN0 [2,]01 > det(A) [1] 0 R isn't consistent, either: > (B <- diag(c(1, NaN))) [,1] [,2] [1,]10 [2,]0 NaN > det(B) [1] NaN Here, DGETRF _does_ succeed, because it does not "see" the trailing NaN in 'B'. So: Should R change to better handle the INFO>0 case? If so, how? Ideally (I think), the proposed change would give NaN for 'A' and 'B' above and 0 for 'C' and 'D' below (both of which really _are_ singular): > (C <- matrix(c(NaN, NaN, 0, 0), 2L, 2L)) [,1] [,2] [1,] NaN0 [2,] NaN0 > det(C) [1] NaN > (D <- t(C)) [,1] [,2] [1,] NaN NaN [2,]00 > det(D) [1] 0 Furthermore, the proposed change should _not_ decrease the performance of determinant(A) for nonsingular 'A' ... For those looking, the relevant C-level function is det_ge_real(), defined in R-devel/src/modules/lapack/Lapack.c (at line 1260 in r83320). Mikael [1] https://github.com/Reference-LAPACK/lapack/blob/master/SRC/dgetrf.f __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[R-pkg-devel] CRAN index replaces ampersand in BugReports URL with HTML entity
Hello, It was raised on the Matrix bug tracker [1] that the BugReports URL on the CRAN index [2] is broken. The URL in our DESCRIPTION file [3] is fine, but contains an ampersand, which is improperly replaced with the corresponding HTML entity in the HTML sources of the index. Hence this does not affect bug.report(package="Matrix"), but _does_ make our bug tracker inaccessible to people accustomed to visiting the CRAN page for links. Can this be repaired? Mikael [1] https://r-forge.r-project.org/tracker/?func=detail=6787_id=61=294 [2] https://cran.r-project.org/package=Matrix [3] https://r-forge.r-project.org/tracker/?atid=294_id=61 __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[Rd] Should package version requirements assume installation from sources?
[Arguably also appropriate for R-package-devel, but posted to R-devel
as the discussion is aimed primarily at "experts" ... ]
We, the authors of Matrix, have encountered a somewhat subtle issue
induced by caching of S4 classes and methods in package namespaces.
The namespaces of three reverse dependent packages (SeuratObject, conText,
mcmcsae) cache the formal definition of our virtual class Matrix (and some
subclasses). For example:
> ns <- asNamespace("SeuratObject")
> grep("^[.]__C__.*Matrix$", names(ns), value = TRUE)
[1] ".__C__dMatrix" ".__C__compMatrix"".__C__AnyMatrix"
[4] ".__C__generalMatrix" ".__C__CsparseMatrix" ".__C__sparseMatrix"
[7] ".__C__dsparseMatrix" ".__C__Matrix"
The cached definition (which includes a _validity method_) is obtained from
the version of Matrix available when the reverse dependent package was built
from sources. For example, if SeuratObject was built under Matrix 1.4-1,
then we get:
> getValidity(ns$.__C__Matrix)
function (object)
{
if (!isTRUE(r <- .Call(Dim_validate, object, "Matrix")))
r
else .Call(dimNames_validate, object)
}
whereas if SeuratObject was built under Matrix >= 1.5-0, then we get:
> getValidity(ns$.__C__Matrix)
function (object)
.Call(Matrix_validate, object)
There are two "questions" here:
1. The symbol 'Matrix_validate' is not defined until Matrix 1.5-0.
Is it necessary, for this reason alone, for SeuratObject to have
'Imports: Matrix (>= 1.5-0)'? Or can SeuratObject continue using
'Imports: Matrix (>= 1.3-3)', at the risk of errors like
> Error: object 'Matrix_validate' not found
(as already seen here: https://stackoverflow.com/questions/73700130)?
Note that this error would not occur for anyone installing SeuratObject
from sources, unless they decide to _downgrade_ Matrix after doing so.
Hence this primarily concerns Windows and macOS where R users would
typically install a binary built by CRAN (i.e., not on their system).
We are aware that package TMB tests in .onLoad() that the current Matrix
version is equal to or greater than the version available at build time,
thus avoiding a "strict" version requirement, but do not want this practice
to spread ...
2. For how long should Matrix retain the superceded 'Dim_validate' and
'dimNames_validate', in order to ensure that "stale" cached validity
methods continue to work?
We hope that this discussion will highlight the potential ramifications
of importing classes and methods from other packages, and having one's
classes and methods imported _by_ other packages, especially for version
requirements.
Mikael, Martin, and Doug
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Subsetting "dspMatrix" without coercion to "matrix"
This seems entirely avoidable, given that there is a relatively simple
formula for converting 2-ary indices [i,j] of S to 1-ary indices k of
S[lower.tri(S, TRUE)]:
k <- i + round(0.5 * (2L * n - j) * (j - 1L)) # for i >= j
I ought to be slightly more precise here: _coercion_ is avoidable,
because we can always map [i,j] to [k], but memory limits are not.
Certainly S@x[k] cannot be arbitrarily long...
At the very least, it would be convenient if the subset were performed
efficiently whenever dimensions would be dropped anyway:
* S[i, ] and S[, j] where i and j are vectors indexing exactly zero or
one rows/columns
* S[i] where i is a matrix of the form cbind(i, j)
This would support, e.g., a memory-efficient 'apply' analogue without
any need for MARGIN...
applySymmetric <- function(X, FUN, ..., simplify = TRUE, check = TRUE) {
if (check && !isSymmetric(X)) {
stop("'X' is not a symmetric matrix.")
}
## preprocessing
ans <- vector("list", n)
for (i in seq_len(n)) {
ans[[i]] <- forceAndCall(1L, FUN, S[i, ], ...)
}
## postprocessing
ans
}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Subsetting "dspMatrix" without coercion to "matrix"
Currently, the class for dense symmetric matrices in packed storage,
"dspMatrix", inherits its subset (i.e., `[`) methods from the "Matrix"
class. As a result, subsetting "dspMatrix" requires coercion to
"matrix". If memory cannot be allocated for this "matrix", then an error
results.
n <- 3L
x <- as.double(seq_len(choose(n + 1L, 2L)))
S <- new("dspMatrix", uplo = "L", x = x, Dim = c(n, n))
S[, 1L]
# Error: vector memory exhausted (limit reached?)
traceback()
# 10: .dsy2mat(dsp2dsy(from))
# 9: asMethod(object)
# 8: as(x, "matrix")
# 7: x[, j = j, drop = TRUE]
# 6: x[, j = j, drop = TRUE]
# 5: eval(call, parent.frame())
# 4: eval(call, parent.frame())
# 3: callGeneric(x, , j = j, drop = TRUE)
# 2: S[, 1L]
# 1: S[, 1L]
This seems entirely avoidable, given that there is a relatively simple
formula for converting 2-ary indices [i,j] of S to 1-ary indices k of
S[lower.tri(S, TRUE)]:
k <- i + round(0.5 * (2L * n - j) * (j - 1L)) # for i >= j
Has the implementation of `[` for class "dspMatrix" been discussed
already elsewhere? If not, shall I report it to Bugzilla as a wishlist item?
FWIW, I encountered this problem while trying to coerce a large "dist"
object to "dspMatrix", expecting that I would be able to safely subset
the result:
setAs(from = "dist", to = "dspMatrix", function(from) {
p <- length(from)
if (p > 0L) {
n <- as.integer(round(0.5 * (1 + sqrt(1 + 8 * p # p = n*(n-1)/2
x <- double(p + n)
i <- 1L + c(0L, cumsum(n:2L))
x[-i] <- from
} else {
n <- 1L
x <- 0
}
new("dspMatrix", uplo = "L", x = x, Dim = c(n, n))
})
But that attempt failed for a entirely different reason. For large
enough n, I would hit a memory limit at the line:
x[-i] <- from
So I guess my second question is: what is problematic about this
benign-seeming subset-assignment?
Mikael
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
