Re: [R-pkg-devel] Package gcc-ASAN issue

[J . Antonio Guzmán Q .]

Hello,

Thanks for your response Kevin. It works using 'inplace_trans()', but it
takes a long time to solve.

I will try to reproduce the error using Dirk's suggestion.

Your guides are appreciated.

AG.

El vie, 16 abr 2021 a las 17:00, Dirk Eddelbuettel ()
escribió:

>
> On 16 April 2021 at 10:27, Kevin Ushey wrote:
> | You can also try to verify locally using e.g. Docker; the rocker
> | images here are immensely helpful:
> |
> | https://github.com/rocker-org/r-devel-san
>
> Thanks but that container image may be stale at times. The daily rebuilds
> by
> Kevin's colleague Winston of his (mega) container r-debug may be better:
>
>https://github.com/wch/r-debug
>
> Dirk
>
> --
> https://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>


-- 
___
J. Antonio Guzmán Q., Ph.D.
Alberta Centre for Earth Observation Sciences
University of Alberta, Canada
jaguzmanq.com
Look my work in Google Scholar
 and/or
ResearchGate 

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package gcc-ASAN issue

[Dirk Eddelbuettel]

On 16 April 2021 at 10:27, Kevin Ushey wrote:
| You can also try to verify locally using e.g. Docker; the rocker
| images here are immensely helpful:
| 
| https://github.com/rocker-org/r-devel-san

Thanks but that container image may be stale at times. The daily rebuilds by
Kevin's colleague Winston of his (mega) container r-debug may be better:

   https://github.com/wch/r-debug

Dirk

-- 
https://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Package gcc-ASAN issue

[Kevin Ushey]

>From https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-ASAN/rTLS/00check.log,
you can see errors of the form:

==3566104==ERROR: AddressSanitizer: stack-use-after-scope on address
0x7fffc2651f80 at pc 0x7f7ba5b635de bp 0x7fffc2651eb0 sp
0x7fffc2651ea0
< ... >

The slides here explain what that means in more detail:
https://llvm.org/devmtg/2016-11/Slides/Buka-StackUseAfterScope.pdf

But, in short, it usually means you took a reference to a variable on
the stack, and then tried to access memory via that reference after
that variable went out-of-scope.

Based on the associated code here:

https://github.com/Antguz/rTLS/blob/449b5c98942a37d4d98d4d2d27ac04caf3cc2cb6/src/knn_rcpp.cpp#L20-L48

A guess is that the issue is related to your use of the non-copying
arma::mat constructors + attempts to transpose them. Could you use
.inplace_trans() instead?

You can also try to verify locally using e.g. Docker; the rocker
images here are immensely helpful:

https://github.com/rocker-org/r-devel-san

Best,
Kevin


On Fri, Apr 16, 2021 at 8:43 AM J. Antonio Guzmán Q.  wrote:
>
> Hello,
>
> We publish in CRAN our package rTLS [1]. The package passed the test
> without problems. However, at some point, additional testing suggests that
> it has issues with 'gcc-ASAN' [2]. For me this error is not informative,
> and thus, I do not have an idea how to start. I have been looking for
> guides or help on the web and unsure of this issue's reason or meaning. It
> is not clear to me what to do based on Writing R Extensions in section
> 4.3.3.
>
> When I see the log files [2], the issue seems to be located in a specific
> function. However, not sure why this is in a specific function since the
> based code is also shared with other functions.
>
> Can you guide me or help me with this? I know that my message is not so
> informative, but I do not know how to start to ask about this issue or
> reproduce the error. Here is the code development repository for this
> package [3].
>
> Antonio,
>
> [1]
> https://cran-archive.r-project.org/web/checks/2021/2021-03-30_check_results_rTLS.html
> [2] https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-ASAN/rTLS/00check.log
> [3] https://github.com/Antguz/rTLS
>
> --
> ___
> J. Antonio Guzmán Q., Ph.D.
> Alberta Centre for Earth Observation Sciences
> University of Alberta, Canada
> jaguzmanq.com
> Look my work in Google Scholar
>  and/or
> ResearchGate 
>
> [[alternative HTML version deleted]]
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] Package gcc-ASAN issue

[J . Antonio Guzmán Q .]

Hello,

We publish in CRAN our package rTLS [1]. The package passed the test
without problems. However, at some point, additional testing suggests that
it has issues with 'gcc-ASAN' [2]. For me this error is not informative,
and thus, I do not have an idea how to start. I have been looking for
guides or help on the web and unsure of this issue's reason or meaning. It
is not clear to me what to do based on Writing R Extensions in section
4.3.3.

When I see the log files [2], the issue seems to be located in a specific
function. However, not sure why this is in a specific function since the
based code is also shared with other functions.

Can you guide me or help me with this? I know that my message is not so
informative, but I do not know how to start to ask about this issue or
reproduce the error. Here is the code development repository for this
package [3].

Antonio,

[1]
https://cran-archive.r-project.org/web/checks/2021/2021-03-30_check_results_rTLS.html
[2] https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-ASAN/rTLS/00check.log
[3] https://github.com/Antguz/rTLS

-- 
___
J. Antonio Guzmán Q., Ph.D.
Alberta Centre for Earth Observation Sciences
University of Alberta, Canada
jaguzmanq.com
Look my work in Google Scholar
 and/or
ResearchGate 

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Problems with too much testing

[Steven Scott]

One of the basic principles of testing is "test interface, not
implementation."  Tests that violate this principle become "change detector
tests" instead of "correctness tests" and essentially prevent any
improvements to the code.  In C++ objects have "public" and "private"
methods, and the testing diktat is to test the public ones.  I hit this
issue a lot at Google when internal clients wrote change detector tests
effectively fencing in my tools.  My solution was to point out the
problematic tests and ask them not to do that.

The intent of R as a language is to be fast, lightweight, and flexible.  In
the interest of keeping it that way for users, it has grown large and
cumbersome for package authors.  If we try to tack on a "public/private"
object layout I'm afraid that would be yet another burden on package
authors.  The most obvious place I see to do this is in the NAMESPACE file,
which is already an ugly beast.  So while I agree with Duncan's
characterization of the problem, to my ears the "social solution" is the
cleanest one available, if we're talking about formal mechanisms.

Informally, both C++ and python use underscores in variable names to
suggest private data members.  You could just prefix "PRIVATE" to the name
of all the data members you don't want tested, where "PRIVATE" might become
"." or "_", or some other character.  I would argue against using
underscores, but I think I've already lost that battle.

On Fri, Apr 16, 2021 at 7:53 AM J C Nash  wrote:

> I'm generally in accord with Duncan on this. There are inevitably
> situations where general
> rules don't apply. Our challenge is to find practical ways to keep the
> overall workload of
> all participants in the process to a minimum.
>
> JN
>
> On 2021-04-16 10:18 a.m., Duncan Murdoch wrote:
> > On 16/04/2021 9:49 a.m., J C Nash wrote:
> >> Another approach is to change the responsibility.
> >>
> >> My feeling is that tests in the TESTING package should be modifiable by
> the maintainer of
> >> the TESTED package, with both packages suspended if the two maintainers
> cannot agree. We
> >> need to be able to move forward when legacy behaviour is outdated or
> just plain wrong. Or,
> >> in the case that I find affects me, when improvements in iterative
> schemes change iterates
> >> slightly. My guess is that Duncan's example is a case in point.
> >>
> >> I doubt this will ever occur, as it doesn't seem to be the R way.
> However, I do know that
> >> improvements in methods are not going to CRAN in some cases.
> >
> > In the cases I've been involved with the authors of the testing package
> have accepted suggested changes when I've made
> > them:  I think that's also part of "the R way".  However, this takes
> time for both of us:  I need to understand what
> > they are intending to test before I can suggest a change to it, and they
> need to understand my change before they can
> > decide if it is acceptable, or whether further changes would also be
> necessary.
> >
> > Github helps a lot with this:  if the testing package is there, I can
> quickly reproduce the issue, produce a fix, and
> > send it to the author, who can tweak it if I've set things up properly.
> >
> > For the kinds of changes you're making, I suspect relaxing a tolerance
> would often be enough, though if you switch
> > algorithms and record that in your results, the testing package may need
> to replace reference values.  I think I'd be
> > uncomfortable doing that.
> >
> > Duncan Murdoch
> >
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Problems with too much testing

[J C Nash]

I'm generally in accord with Duncan on this. There are inevitably situations 
where general
rules don't apply. Our challenge is to find practical ways to keep the overall 
workload of
all participants in the process to a minimum.

JN

On 2021-04-16 10:18 a.m., Duncan Murdoch wrote:
> On 16/04/2021 9:49 a.m., J C Nash wrote:
>> Another approach is to change the responsibility.
>>
>> My feeling is that tests in the TESTING package should be modifiable by the 
>> maintainer of
>> the TESTED package, with both packages suspended if the two maintainers 
>> cannot agree. We
>> need to be able to move forward when legacy behaviour is outdated or just 
>> plain wrong. Or,
>> in the case that I find affects me, when improvements in iterative schemes 
>> change iterates
>> slightly. My guess is that Duncan's example is a case in point.
>>
>> I doubt this will ever occur, as it doesn't seem to be the R way. However, I 
>> do know that
>> improvements in methods are not going to CRAN in some cases.
> 
> In the cases I've been involved with the authors of the testing package have 
> accepted suggested changes when I've made
> them:  I think that's also part of "the R way".  However, this takes time for 
> both of us:  I need to understand what
> they are intending to test before I can suggest a change to it, and they need 
> to understand my change before they can
> decide if it is acceptable, or whether further changes would also be 
> necessary.
> 
> Github helps a lot with this:  if the testing package is there, I can quickly 
> reproduce the issue, produce a fix, and
> send it to the author, who can tweak it if I've set things up properly.
> 
> For the kinds of changes you're making, I suspect relaxing a tolerance would 
> often be enough, though if you switch
> algorithms and record that in your results, the testing package may need to 
> replace reference values.  I think I'd be
> uncomfortable doing that.
> 
> Duncan Murdoch
>

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Problems with too much testing

[Duncan Murdoch]

On 16/04/2021 9:49 a.m., J C Nash wrote:

Another approach is to change the responsibility.

My feeling is that tests in the TESTING package should be modifiable by the 
maintainer of
the TESTED package, with both packages suspended if the two maintainers cannot 
agree. We
need to be able to move forward when legacy behaviour is outdated or just plain 
wrong. Or,
in the case that I find affects me, when improvements in iterative schemes 
change iterates
slightly. My guess is that Duncan's example is a case in point.

I doubt this will ever occur, as it doesn't seem to be the R way. However, I do 
know that
improvements in methods are not going to CRAN in some cases.


In the cases I've been involved with the authors of the testing package 
have accepted suggested changes when I've made them:  I think that's 
also part of "the R way".  However, this takes time for both of us:  I 
need to understand what they are intending to test before I can suggest 
a change to it, and they need to understand my change before they can 
decide if it is acceptable, or whether further changes would also be 
necessary.


Github helps a lot with this:  if the testing package is there, I can 
quickly reproduce the issue, produce a fix, and send it to the author, 
who can tweak it if I've set things up properly.


For the kinds of changes you're making, I suspect relaxing a tolerance 
would often be enough, though if you switch algorithms and record that 
in your results, the testing package may need to replace reference 
values.  I think I'd be uncomfortable doing that.


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Problems with too much testing

[J C Nash]

Another approach is to change the responsibility.

My feeling is that tests in the TESTING package should be modifiable by the 
maintainer of
the TESTED package, with both packages suspended if the two maintainers cannot 
agree. We
need to be able to move forward when legacy behaviour is outdated or just plain 
wrong. Or,
in the case that I find affects me, when improvements in iterative schemes 
change iterates
slightly. My guess is that Duncan's example is a case in point.

I doubt this will ever occur, as it doesn't seem to be the R way. However, I do 
know that
improvements in methods are not going to CRAN in some cases.

JN

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Problems with too much testing

[Rainer Krug]

> On 16 Apr 2021, at 14:06, Duncan Murdoch  wrote:
> 
> On 16/04/2021 7:54 a.m., Hugh Parsonage wrote:
>> My 2c is that if a package has actually tested something like that
>> (say, the ordering of a list's elements), then it is 100% likely that
>> a script out there depends on that behaviour too. 
> 
> I suspect this test is not intentional, and it really wouldn't make sense for 
> a package to depend on the order of named list components.

You could randomise the order….


> 
> In other words, the
>> change is not inconsequential.  R users are in debt to package
>> developers but I think it behoves us to take special care not to
>> underestimate the work involved when a package's behaviour changes.
> 
> There's a difference between internal implementation changes and external 
> behaviour changes.  For example, there are clear warnings not to use 
> un-exported functions (via :::), and a package that did that would not be 
> accepted on CRAN.
> 
> What I'm asking for are suggestions for ways to hide internal layouts of 
> objects from users.  That seems a lot harder.
> 
> Duncan Murdoch
> 
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel


__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] gcc-ASAN issue

[J . Antonio Guzmán Q .]

Hello,

We publish in CRAN our package rTLS [1]. The package passed the test
without problems. However, at some point, additional testing suggests that
it has issues with 'gcc-ASAN' [2]. For me this error is not informative,
and thus, I do not have an idea how to start. I have been looking for
guides or help on the web and unsure of this issue's reason or meaning. It
is not clear to me what to do based on Writing R Extensions in section
4.3.3.

When I see the log files [2], the issue seems to be located in a specific
function. However, not sure why this is in a specific function since the
based code is also shared with other functions.

Can you guide me or help me with this? I know that my message is not so
informative, but I do not know how to start to ask about this issue. Here
is the code development repository for this package [3].
Antonio,

[1]
https://cran-archive.r-project.org/web/checks/2021/2021-03-30_check_results_rTLS.html
[2] https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-ASAN/rTLS/00check.log
[3] https://github.com/Antguz/rTLS

-- 
___
J. Antonio Guzmán Q., Ph.D.
Alberta Centre for Earth Observation Sciences
University of Alberta, Canada
jaguzmanq.com
Look my work in Google Scholar
 and/or
ResearchGate 

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Problems with too much testing

[Duncan Murdoch]

On 16/04/2021 7:54 a.m., Hugh Parsonage wrote:

My 2c is that if a package has actually tested something like that
(say, the ordering of a list's elements), then it is 100% likely that
a script out there depends on that behaviour too. 


I suspect this test is not intentional, and it really wouldn't make 
sense for a package to depend on the order of named list components.


 In other words, the

change is not inconsequential.  R users are in debt to package
developers but I think it behoves us to take special care not to
underestimate the work involved when a package's behaviour changes.


There's a difference between internal implementation changes and 
external behaviour changes.  For example, there are clear warnings not 
to use un-exported functions (via :::), and a package that did that 
would not be accepted on CRAN.


What I'm asking for are suggestions for ways to hide internal layouts of 
objects from users.  That seems a lot harder.


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Problems with too much testing

[Duncan Murdoch]

On 16/04/2021 7:40 a.m., brodie gaslam wrote:

An all.equal method?  This might not work with 3rd edition though
(untested), and I'm not sure what method registration requirements
would exist for the user.


It already has one, but it isn't used by testthat 3.

Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Problems with too much testing

[Hugh Parsonage]

My 2c is that if a package has actually tested something like that
(say, the ordering of a list's elements), then it is 100% likely that
a script out there depends on that behaviour too.  In other words, the
change is not inconsequential.  R users are in debt to package
developers but I think it behoves us to take special care not to
underestimate the work involved when a package's behaviour changes.

On Fri, 16 Apr 2021 at 20:09, Duncan Murdoch  wrote:
>
> I'm updating the rgl package, and have come across the following problem.
>
> Some packages that depend on rgl and do careful testing are detecting
> inconsequential changes.  A typical case is the nat package, which has
> extensive testing, some of which comes down to checking results from rgl
> functions using testthat::is_equal().  I rewrote some parts of the rgl
> code, and so some rgl objects differ in irrelevant ways.  For example,
>
>obj <- list(x = NULL)
>
> differs from
>
>obj <- list()
>obj[["x"]] <- NULL
>
> (the first is length 1, the second ends up with no entries).  All tests
> in rgl would be like
>
>if (is.null(obj[["x"]])) ...
>
> so the tests evaluate the same, but testthat::is_equal() doesn't return
> TRUE, because the objects are different.
>
> Another example would be
>
>obj <- list()
>obj$a <- 1
>obj$b <- 2
>
> which differs from
>
>obj <- list()
>obj$b <- 2
>obj$a <- 1
>
> in the order of the components, but since rgl always accessed them by
> name, that makes no difference.
>
> So what I'm looking for is a strategy to hide internal implementation
> details.  (I'm using a compare_proxy() method now which standardizes the
> objects, but that seems very specific to testthat version 3.  I'd like
> ideas for something more robust.)
>
> Duncan Murdoch
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Problems with too much testing

[brodie gaslam via R-package-devel]

An all.equal method?  This might not work with 3rd edition though
(untested), and I'm not sure what method registration requirements
would exist for the user.

Best,

B.

On Friday, April 16, 2021, 6:09:24 AM EDT, Duncan Murdoch 
 wrote: 





I'm updating the rgl package, and have come across the following problem.

Some packages that depend on rgl and do careful testing are detecting 
inconsequential changes.  A typical case is the nat package, which has 
extensive testing, some of which comes down to checking results from rgl 
functions using testthat::is_equal().  I rewrote some parts of the rgl 
code, and so some rgl objects differ in irrelevant ways.  For example,

  obj <- list(x = NULL)

differs from

  obj <- list()
  obj[["x"]] <- NULL

(the first is length 1, the second ends up with no entries).  All tests 
in rgl would be like

  if (is.null(obj[["x"]])) ...

so the tests evaluate the same, but testthat::is_equal() doesn't return 
TRUE, because the objects are different.

Another example would be

  obj <- list()
  obj$a <- 1
  obj$b <- 2

which differs from

  obj <- list()
  obj$b <- 2
  obj$a <- 1

in the order of the components, but since rgl always accessed them by 
name, that makes no difference.

So what I'm looking for is a strategy to hide internal implementation 
details.  (I'm using a compare_proxy() method now which standardizes the 
objects, but that seems very specific to testthat version 3.  I'd like 
ideas for something more robust.)

Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] Problems with too much testing

[Duncan Murdoch]

I'm updating the rgl package, and have come across the following problem.

Some packages that depend on rgl and do careful testing are detecting 
inconsequential changes.  A typical case is the nat package, which has 
extensive testing, some of which comes down to checking results from rgl 
functions using testthat::is_equal().  I rewrote some parts of the rgl 
code, and so some rgl objects differ in irrelevant ways.  For example,


  obj <- list(x = NULL)

differs from

  obj <- list()
  obj[["x"]] <- NULL

(the first is length 1, the second ends up with no entries).  All tests 
in rgl would be like


  if (is.null(obj[["x"]])) ...

so the tests evaluate the same, but testthat::is_equal() doesn't return 
TRUE, because the objects are different.


Another example would be

  obj <- list()
  obj$a <- 1
  obj$b <- 2

which differs from

  obj <- list()
  obj$b <- 2
  obj$a <- 1

in the order of the components, but since rgl always accessed them by 
name, that makes no difference.


So what I'm looking for is a strategy to hide internal implementation 
details.  (I'm using a compare_proxy() method now which standardizes the 
objects, but that seems very specific to testthat version 3.  I'd like 
ideas for something more robust.)


Duncan Murdoch

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel