Re: [R-pkg-devel] Additional issues: Intel segfault

2024-03-10 Thread Murray Efford
For the record - the original 'Additional issue' (Intel segfault on exit) has 
spontaneously disappeared, for both packages secrdesign and ipsecr (I also 
found the error in some other packages that used RcppArmadillo, but haven't 
rechecked them).

From: Ivan Krylov 
Sent: Saturday, 2 March 2024 20:45
To: Murray Efford 
Cc: R-package-devel@r-project.org 
Subject: Re: [R-pkg-devel] Additional issues: Intel segfault

� Sat, 2 Mar 2024 02:07:47 +
Murray Efford  �:

> Gabor suggested 
> https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fr-hub%2Frhub2=05%7C02%7Cmurray.efford%40otago.ac.nz%7Cf74f0a4e96b14115b95a08dc3a8cc260%7C0225efc578fe4928b1579ef24809e9ba%7C0%7C0%7C638449623464096131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=WBz4HzJ%2FKlAKdRimgsiHPx3bIoB2Nvl%2FUQRdh6Ldv%2Fs%3D=0<https://github.com/r-hub/rhub2>
>  and that worked like a
> charm. A check there on the Intel platform found no errors in my
> present version of secrdesign, so I'll resubmit with confidence.

Thank you for letting me know! Having this as a container simplifies a
lot of things.

--
Best regards,
Ivan

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


Re: [R-pkg-devel] Additional issues: Intel segfault

2024-03-01 Thread Ivan Krylov via R-package-devel
В Sat, 2 Mar 2024 02:07:47 +
Murray Efford  пишет:

> Gabor suggested https://github.com/r-hub/rhub2 and that worked like a
> charm. A check there on the Intel platform found no errors in my
> present version of secrdesign, so I'll resubmit with confidence.

Thank you for letting me know! Having this as a container simplifies a
lot of things.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


Re: [R-pkg-devel] Additional issues: Intel segfault

2024-03-01 Thread Murray Efford
Thanks, Ivan, for looking into this and providing some reassurance. Gabor 
suggested https://github.com/r-hub/rhub2 and that worked like a charm. A check 
there on the Intel platform found no errors in my present version of 
secrdesign, so I'll resubmit with confidence. The original error remains a 
mystery, but not one I need to pursue.
Murray

From: Ivan Krylov 
Sent: Friday, 1 March 2024 21:46
To: Murray Efford 
Cc: R-package-devel@r-project.org 
Subject: Re: [R-pkg-devel] Additional issues: Intel segfault

В Fri, 1 Mar 2024 07:42:01 +
Murray Efford  пишет:

> R CMD check suggests it is most likely in the Examples for
> 'validate', but all code there is wrapped in \dontrun{}.

The crash happens after q('no'), suggesting a corruption in the heap or
in the R memory manager. At least it's a null pointer being
dereferenced and not a 0xRANDOM_LOOKING_NUMBER: this limits the impact
of the problem.

I don't know if anyone created an easily reproducible container with an
Intel build of R (there's https://hub.docker.com/r/intel/oneapi, but
aren't the compilers themselves supposed to be not redistributable?),
so you will most likely have to follow
https://www.stats.ox.ac.uk/pub/bdr/Intel/README.txt and
https://cran.r-project.org/doc/manuals/r-devel/R-admin.html#Intel-compilers
manually, compiling R using Intel compilers yourself in order to
reproduce this.

I think it would be great if CRAN checking machines used a just-in-time
debugger to provide C-level backtraces at the place of the crash. For
Windows, such a utility does exist [*], but I recently learned that the
glibc `catchsegv` program (and most other similar programs) used to
perform shared object preloading (before being thrown out of the
codebase altogether), which is more intrusive than it could be. A proof
of concept using GDB on Linux can be shown to work:

R -d gdb \
 --debugger-args='-batch -ex run -ex bt -ex c -ex q' \
 -e '
  Rcpp::sourceCpp(code =
   "//[[Rcpp::export]]\nvoid rip() { *(double*)(42) = 42; }"
  ); rip()
 '

--
Best regards,
Ivan

[*] https://github.com/jrfonseca/drmingw

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


Re: [R-pkg-devel] Additional issues: Intel segfault

2024-03-01 Thread Ivan Krylov via R-package-devel
В Fri, 1 Mar 2024 07:42:01 +
Murray Efford  пишет:

> R CMD check suggests it is most likely in the Examples for
> 'validate', but all code there is wrapped in \dontrun{}.

The crash happens after q('no'), suggesting a corruption in the heap or
in the R memory manager. At least it's a null pointer being
dereferenced and not a 0xRANDOM_LOOKING_NUMBER: this limits the impact
of the problem.

I don't know if anyone created an easily reproducible container with an
Intel build of R (there's https://hub.docker.com/r/intel/oneapi, but
aren't the compilers themselves supposed to be not redistributable?),
so you will most likely have to follow
https://www.stats.ox.ac.uk/pub/bdr/Intel/README.txt and
https://cran.r-project.org/doc/manuals/r-devel/R-admin.html#Intel-compilers
manually, compiling R using Intel compilers yourself in order to
reproduce this.

I think it would be great if CRAN checking machines used a just-in-time
debugger to provide C-level backtraces at the place of the crash. For
Windows, such a utility does exist [*], but I recently learned that the
glibc `catchsegv` program (and most other similar programs) used to
perform shared object preloading (before being thrown out of the
codebase altogether), which is more intrusive than it could be. A proof
of concept using GDB on Linux can be shown to work:

R -d gdb \
 --debugger-args='-batch -ex run -ex bt -ex c -ex q' \
 -e '
  Rcpp::sourceCpp(code =
   "//[[Rcpp::export]]\nvoid rip() { *(double*)(42) = 42; }"
  ); rip()
 '

-- 
Best regards,
Ivan

[*] https://github.com/jrfonseca/drmingw

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


Re: [R-pkg-devel] Additional Issues: Intel

2024-01-17 Thread Tomas Kalibera



On 1/17/24 09:41, Ivan Krylov via R-package-devel wrote:

В Wed, 17 Jan 2024 10:30:36 +1100
Hugh Parsonage  пишет:


I am unable to immediately see where in the test suite this error has
occurred.

Without testthat, you would have gotten a line by line printout of the code, 
letting you pinpoint the (top-level) place of the crash. With
testthat, you will need a more verbose reporter that would print tests
as they are executed to find out which test causes the crash.


The only hunch I have is that the package uses C code and includes
structs with arrays on the stack, which perhaps are excessive for the
Intel check machine, but am far from confident that's the issue.

According to GNU cflow, your only recursive C functions are
getListElement (from getListElement.c) and nthOffset (from Offset.c),
but the recursion seems bounded in both cases.

I've tried looking for variable-length arrays in your code using a
Coccinelle patch, but found none. If you had variable-bounded recursion
or variable-length stack arrays (VLA or alloca()), it would be prudent
to use R_CheckStack() or R_CheckStack2(size_of_VLA), but your C code
contains neither, so there's no obvious culprit. If you know about
R-level recursion happening in your code and have a way to reduce it,
that might help too.

Otherwise, it's time to install Intel Everything and reproduce and
debug the problem the hard way.


You could also try debugging using your toolchain, but with reduced 
stack size (e.g. ulimit -s). If you can make the error appear with a 
smaller but still reasonable stack size, chances are it is due to the 
same underlying problem.


Tomas





__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


Re: [R-pkg-devel] Additional Issues: Intel

2024-01-17 Thread Ivan Krylov via R-package-devel
В Wed, 17 Jan 2024 10:30:36 +1100
Hugh Parsonage  пишет:

> I am unable to immediately see where in the test suite this error has
> occurred.

Without testthat, you would have gotten a line by line printout of the code, 
letting you pinpoint the (top-level) place of the crash. With
testthat, you will need a more verbose reporter that would print tests
as they are executed to find out which test causes the crash.

> The only hunch I have is that the package uses C code and includes
> structs with arrays on the stack, which perhaps are excessive for the
> Intel check machine, but am far from confident that's the issue.

According to GNU cflow, your only recursive C functions are
getListElement (from getListElement.c) and nthOffset (from Offset.c),
but the recursion seems bounded in both cases.

I've tried looking for variable-length arrays in your code using a
Coccinelle patch, but found none. If you had variable-bounded recursion
or variable-length stack arrays (VLA or alloca()), it would be prudent
to use R_CheckStack() or R_CheckStack2(size_of_VLA), but your C code
contains neither, so there's no obvious culprit. If you know about
R-level recursion happening in your code and have a way to reduce it,
that might help too.

Otherwise, it's time to install Intel Everything and reproduce and
debug the problem the hard way.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


Re: [R-pkg-devel] Additional Issues: Intel

2024-01-16 Thread Vladimir Dergachev




On Wed, 17 Jan 2024, Hugh Parsonage wrote:


My package grattan fails the Intel[1] check with

 Error: segfault from C stack overflow

I am unable to immediately see where in the test suite this error has
occurred.  I seek advice on how to fix this error.  The only hunch I
have is that the package uses C code and includes structs with arrays
on the stack, which perhaps are excessive for the Intel check machine,
but am far from confident that's the issue.  The repository is at



Two possibilities to look into:

   * your structures on the stack are large. Don't do this ! Your code 
might run faster and would be easier to debug if you use regular memory 
allocation instead. Since R does fair number of memory allocation calls 
itself, the extra overhead from your calls will not be that noticeable.


   * your stuctures are small, but you have a recursive function that is 
called too often. In this case, the solution is to reimplement the 
recurrence without doing function calls (using a loop, for example). Some 
recurrences can be implemented without using any accumulating state. 
Others need it and you can use heap memory for that.


best

Vladimir Dergachev



[1]https://www.stats.ox.ac.uk/pub/bdr/Intel/grattan.out

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel



__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel