[
https://issues.apache.org/jira/browse/ARROW-9616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-9616:
--------------------------
Description:
The next version of R might enable LTO on Windows, i.e. R packages will be
compiled with {{-flto}} by default. This works out of the box for most
packages, but for arrow, the linker crashes as below.
{code}
C:/rtools40/mingw64/bin/g++ -shared -O2 -Wall -mfpmath=sse -msse2
-mstackrealign -flto -s -static-libgcc -o arrow.dll tmp.def array.o
array_from_vector.o array_to_vector.o arraydata.o arrowExports.o buffer.o
chunkedarray.o compression.o compute.o csv.o dataset.o datatype.o expression.o
feather.o field.o filesystem.o imports.o io.o json.o memorypool.o message.o
parquet.o py-to-r.o recordbatch.o recordbatchreader.o recordbatchwriter.o
scalar.o schema.o symbols.o table.o threadpool.o -L../windows//lib-8.3.0/x64
-L../windows//lib/x64 -lparquet -larrow_dataset -larrow -lthrift -lsnappy -lz
-lzstd -llz4 -lbcrypt -lpsapi -lcrypto -lcrypt32 -lws2_32
-LC:/PROGRA~1/R/R-devel/bin/x64 -lR
lto1.exe: internal compiler error: in add_symbol_to_partition_1, at
lto/lto-partition.c:153
libbacktrace could not find executable to open
Please submit a full bug report,
with preprocessed source if appropriate.
See <[https://github.com/r-windows]> for instructions.
lto-wrapper.exe: fatal error: C:\rtools40\mingw64\bin\g++.exe returned 1 exit
status
compilation terminated.
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
error: lto-wrapper failed
{code}
You can reproduce this in R on Windows for example like so:
{code:r}
dir.create("~/.R")
writeLines("CXXFLAGS=-flto", file = "~/.R/Makevars")
install.packages("arrow", type = 'source')
{code}
I am not sure if this is a bug in the toolchain, or in arrow. I tried with both
gcc-8.3.0 and gcc-9.3.0, and the result is the same. I did find [this
issue|https://github.com/cycfi/elements/pull/56] in another project which
suggests to enable `INTERPROCEDURAL_OPTIMIZATION` in cmake, when mixing lto
code with non-lto code (which is the case when we only build the r bindings
with lto, but not the c++ library).
was:
The next version of R might enable LTO on Windows, i.e. R packages will be
compiled with {{-flto}} by default. This works out of the box for most
packages, but for arrow, the linker crashes as below.
{code}
C:/rtools40/mingw64/bin/g++ -shared -O2 -Wall -mfpmath=sse -msse2
-mstackrealign -flto -s -static-libgcc -o arrow.dll tmp.def array.o
array_from_vector.o array_to_vector.o arraydata.o arrowExports.o buffer.o
chunkedarray.o compression.o compute.o csv.o dataset.o datatype.o expression.o
feather.o field.o filesystem.o imports.o io.o json.o memorypool.o message.o
parquet.o py-to-r.o recordbatch.o recordbatchreader.o recordbatchwriter.o
scalar.o schema.o symbols.o table.o threadpool.o -L../windows//lib-8.3.0/x64
-L../windows//lib/x64 -lparquet -larrow_dataset -larrow -lthrift -lsnappy -lz
-lzstd -llz4 -lbcrypt -lpsapi -lcrypto -lcrypt32 -lws2_32
-LC:/PROGRA~1/R/R-devel/bin/x64 -lR
lto1.exe: internal compiler error: in add_symbol_to_partition_1, at
lto/lto-partition.c:153
libbacktrace could not find executable to open
Please submit a full bug report,
with preprocessed source if appropriate.
See <[https://github.com/r-windows]> for instructions.
lto-wrapper.exe: fatal error: C:\rtools40\mingw64\bin\g++.exe returned 1 exit
status
compilation terminated.
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
error: lto-wrapper failed
{code}
You can easily produce this for example like this:
{code:r}
dir.create("~/.R")
writeLines("CXXFLAGS=-flto", file = "~/.R/Makevars")
install.packages("arrow", type = 'source')
{code}
I am not sure if this is a bug in the toolchain, or in arrow. I tried with both
gcc-8.3.0 and gcc-9.3.0, and the result is the same. I did find [this
issue|https://github.com/cycfi/elements/pull/56] in another project which
suggests to enable `INTERPROCEDURAL_OPTIMIZATION` in cmake, when mixing lto
code with non-lto code (which is the case when we only build the r bindings
with lto, but not the c++ library).
> [C++] Support LTO for R
> -----------------------
>
> Key: ARROW-9616
> URL: https://issues.apache.org/jira/browse/ARROW-9616
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
> Affects Versions: 1.0.0
> Reporter: Jeroen
> Assignee: Neal Richardson
> Priority: Major
>
> The next version of R might enable LTO on Windows, i.e. R packages will be
> compiled with {{-flto}} by default. This works out of the box for most
> packages, but for arrow, the linker crashes as below.
> {code}
> C:/rtools40/mingw64/bin/g++ -shared -O2 -Wall -mfpmath=sse -msse2
> -mstackrealign -flto -s -static-libgcc -o arrow.dll tmp.def array.o
> array_from_vector.o array_to_vector.o arraydata.o arrowExports.o buffer.o
> chunkedarray.o compression.o compute.o csv.o dataset.o datatype.o
> expression.o feather.o field.o filesystem.o imports.o io.o json.o
> memorypool.o message.o parquet.o py-to-r.o recordbatch.o recordbatchreader.o
> recordbatchwriter.o scalar.o schema.o symbols.o table.o threadpool.o
> -L../windows//lib-8.3.0/x64 -L../windows//lib/x64 -lparquet -larrow_dataset
> -larrow -lthrift -lsnappy -lz -lzstd -llz4 -lbcrypt -lpsapi -lcrypto
> -lcrypt32 -lws2_32 -LC:/PROGRA~1/R/R-devel/bin/x64 -lR
> lto1.exe: internal compiler error: in add_symbol_to_partition_1, at
> lto/lto-partition.c:153
> libbacktrace could not find executable to open
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See <[https://github.com/r-windows]> for instructions.
> lto-wrapper.exe: fatal error: C:\rtools40\mingw64\bin\g++.exe returned 1
> exit status
> compilation terminated.
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
> error: lto-wrapper failed
> {code}
> You can reproduce this in R on Windows for example like so:
> {code:r}
> dir.create("~/.R")
> writeLines("CXXFLAGS=-flto", file = "~/.R/Makevars")
> install.packages("arrow", type = 'source')
> {code}
> I am not sure if this is a bug in the toolchain, or in arrow. I tried with
> both gcc-8.3.0 and gcc-9.3.0, and the result is the same. I did find [this
> issue|https://github.com/cycfi/elements/pull/56] in another project which
> suggests to enable `INTERPROCEDURAL_OPTIMIZATION` in cmake, when mixing lto
> code with non-lto code (which is the case when we only build the r bindings
> with lto, but not the c++ library).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
