On 1/19/24 15:18, Serge wrote:
This post is a continuation of the post *[R-pkg-devel] Does
dependencies up to date on the pretest CRAN infrastructure*
I made more (unsuccessful) tries:
- I installed a Windows 11 version in a VM on my compuiter and try to
buid the MixAll package using Rtools42 and Rtools43 (it's quite easy,
and funny, to do it on windows: you have just to rename C:\rtools42 as
C:\rtools43).
There should be no renaming, instead, one should use R 4.2.x with
Rtools42 and R 4.3.x (or current R-devel) with Rtools43. All of these
can co-exist (be installed at the same time). Mixing the two could lead
to different failures. I understand you want to test different versions
of GCC, but to do that reliable you would have to rebuild the rest of
Rtools with that, or just the part that is needed by the package.
The result is that MixAll is build using the 4.2 version and the buid
failed using the 4.3 version.
Please make sure to use Rtools43 (the real one) with R 4.3.
- I installed the version 12.3 of gcc on ubuntu (the same version used
on windows) and could build the package without problem
- Inspecting the log of the Rtools4.3
(https://cran.r-project.org/bin/windows/Rtools/rtools43/news.html) and
g++12..3 (https://gcc.gnu.org/gcc-12/changes.html) does not give insight.
- The package is dependent of the rtkore package which use extensive
use of templated class. As rtkore is a port for R of stk++, I try to
compile the stk++ library on windows (using g++12). All tests are
compiled without any troubles.
These attempts eliminate some causes, but don't give me any insight
why MixAll (and blockcluster) failed to be build on the Windows-devel
platform. It seems related to Rtools43. Does anyone else (using for
exemple Rcppeigen) is experiencing this problem ?
If this is GCC running out of memory on Windows but not Linux, when
there really should be enough of memory available (i.e. due to the
problem that Uwe described, maybe the internal GC in GCC is asking for
too much memory for its heap given that the OS is not overcomiting), you
can try narrowing it down to a standalone C++ program (independent on R,
Rtools, R packages, compilable on both Windows and Linux with disabled
optimizations, etc) that still exhibits the problem and then submitting
it in a bug report to GCC bugzilla.
In such case, it could be that some heuristics in the collector could be
improved.
If you have such standalone example, it would be easier to test
different versions of GCC or even bisect to a concrete GCC version. You
might also compare memory usage when compiling and cross-compiling.
You might also be able to find a work-around for your package by
disabling some compiler optimizations. Also if you can narrow this down
to a concrete GCC optimization then it would help to mention that in the
bug report.
Certainly all of this would be out of scope of R-(pkg)-devel, this would
rather be a GCC/Windows thing.
Tomas
Serge
Le 14/01/2024 à 18:50, Uwe Ligges a écrit :
On 13.01.2024 15:01, Uwe Ligges wrote:
Fascinating, now it worked with the latest winbuilder submission 3
times in a row when I checked it manually. So maybe Ivan was right
and there was a very demanding set of other packages compiling at
the same time?
I don't know.
Serge, Can you somply submit your latest winbuilder upload to CRAN?
Really, I inspected some more. The underlying issue is simple:
The C++ compiler used under Windows asks for precomitted memory. If
several processes are running at the same time, a lot of memory is
precomitted. And Windows does not use it for other processes, even if
almost nothing is actually used.
So while the used memory may be around 50GB, all of the rest (of 756
GB including swap space) may have been precomitted (but unused) and
new processes failed to start correctly. Grrrr.
Best,
Uwe Ligges
Best,
Uwe Ligges
On 13.01.2024 14:12, Uwe Ligges wrote:
I can take a look, but not sure if I get to it before monday.
I haven't seen it for any other packages recently.
My suspicion is currently a strange mix of cmd.exe and sh.exe
calls. But this is a very wild guess.
Best,
Uwe
On 13.01.2024 14:08, Uwe Ligges wrote:
On 13.01.2024 10:10, Ivan Krylov via R-package-devel wrote:
В Fri, 12 Jan 2024 21:19:00 +0100
Serge <serge.iovl...@stkpp.org> пишет:
After somme minor midficiations, I make a try on the winbuilder
site.
I was able to build the archive with the static library
but I get again a Bad address error. You can have a look to
https://win-builder.r-project.org/bw47qsMX3HTd/00install.out
I think that Win-Builder is running out of memory. It took some
experimenting, but I was able to reproduce something like this using
the following:
1. Set the swap file in the Windows settings to minimal recommended
size and disable its automatic growth
2. Write and run a program that does malloc(LARGE_NUMBER);
getchar();
so that almost all physical memory is allocated
3. Run gcc -DFOO=`/path/to/Rscript -e 'some script'` & many times
I got a lot of interesting errors, including the "Bad address":
Warnings:
1: .getGeneric(f, , package) : internal error -4 in R_decompress1
2: package "methods" in options("defaultPackages") was not found
0 [main] bash (2892) child_copy: cygheap read copy failed,
0x0..0x800025420, done 0, windows pid 2892, Win32 error 299
0 [main] bash (3256) C:\rtools43\usr\bin\bash.exe: *** fatal
error in
forked process - MEM_COMMIT failed, Win32 error 1455
-bash: fork: retry: Resource temporarily unavailable
-bash: R-devel/bin/Rscript.exe: Bad address
The above indeed happens if not sufficient memory would be available.
Important to know: This includes unused but committed memory which
may be a lot.
But I doubt it is the case on winbuilder as the machines has 256GB
or more (depending in the machine) and additionally 500GB swap
space on SSD.
Best,
Uwe
Your package is written in C++, but that by itself shouldn't
disqualify
it. On my Linux computer, /usr/bin/time R -e
'install.packages("MixAll")' says that the installation takes
slightly
less than a gigabyte of memory ("912516maxresident k"), which is par
the course for such packages. (My small Rcpp-using package takes
approximately half a gigabyte by the same metric.)
I'm still not 100% sure (if Win-Builder is running out of memory,
why
are you seeing "Bad address" only and not the rest of the carnage?),
but I'm not seeing a problem with your package, either. If EFAULT is
Cygwin's way of saying "I caught a bad pointer in your system call"
(which, I must stress, is happening inside /bin/sh, not your package
or even R at all), it's not impossible that Win-Builder is having
hardware problems. Unfortunately, they take a lot of effort and
downtime to diagnose and could be hiding anywhere from RAM to the
power
supply.
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
