tdhock opened a new issue, #35566: URL: https://github.com/apache/arrow/issues/35566
### Describe the bug, including details regarding any error messages, version, and platform. Hi! I'm trying to install libarrow C++ from source, and then install R package from source. I expected that this should be possible without having to manually set environment variables ARROW_PARQUET and ARROW_CSV, but I found that I have to set them in order for R package installation to succeed, so I believe this may be a bug. maybe similar to https://github.com/apache/arrow/issues/27825 ? In detail I have cloned arrow from github, and built C++ libarrow from source via code below, ``` (arrow) tdhock@maude-MacBookPro:~/arrow-git/cpp/build(main)$ CC=$HOME/bin/gcc CXX=$HOME/bin/g++ cmake .. --preset ninja-debug-basic -DCMAKE_INSTALL_PREFIX=$HOME -DARROW_CXXFLAGS=-march=core2 -DARROW_PARQUET=ON Preset CMake variables: ARROW_BUILD_INTEGRATION="ON" ARROW_BUILD_STATIC="OFF" ARROW_BUILD_TESTS="ON" ARROW_COMPUTE="ON" ARROW_CSV="ON" ARROW_DATASET="ON" ARROW_EXTRA_ERROR_CONTEXT="ON" ARROW_FILESYSTEM="ON" ARROW_JSON="ON" ARROW_WITH_RE2="OFF" ARROW_WITH_UTF8PROC="OFF" CMAKE_BUILD_TYPE="Debug" -- Building using CMake version: 3.22.1 -- Arrow version: 13.0.0 (full: '13.0.0-SNAPSHOT') -- Arrow SO version: 1300 (full: 1300.0.0) ... -- Outputting build configuration summary to /home/tdhock/arrow-git/cpp/build/cmake_summary.json -- Configuring done -- Generating done -- Build files have been written to: /home/tdhock/arrow-git/cpp/build (arrow) tdhock@maude-MacBookPro:~/arrow-git/cpp/build(main)$ cmake --build . [0/1] Re-running CMake... -- Building using CMake version: 3.22.1 -- Arrow version: 13.0.0 (full: '13.0.0-SNAPSHOT') -- Arrow SO version: 1300 (full: 1300.0.0) ... -- Outputting build configuration summary to /home/tdhock/arrow-git/cpp/build/cmake_summary.json -- Configuring done -- Generating done -- Build files have been written to: /home/tdhock/arrow-git/cpp/build [1/365] Building CXX object src/arrow/CMakeFiles/arrow_objlib.dir/io/buffered.cc.o [2/365] Building CXX object src/arrow/CMakeFiles/arrow_objlib.dir/io/caching.cc.o ... [364/365] Building CXX object src/parquet/CMakeFiles/parquet-arrow-test.dir/arrow/arrow_reader_writer_test.cc.o [365/365] Linking CXX executable debug/parquet-arrow-test (arrow) tdhock@maude-MacBookPro:~/arrow-git/cpp/build(main)$ cmake --install . -- Install configuration: "DEBUG" -- Installing: /home/tdhock/lib/cmake/Arrow/FindThriftAlt.cmake -- Installing: /home/tdhock/include/arrow/util/config.h ... -- Installing: /home/tdhock/include/parquet/encryption/test_in_memory_kms.h -- Installing: /home/tdhock/include/parquet/encryption/two_level_cache_with_expiration.h ``` The code above builds libarrow and installs it under my home directory. The code below then tries to install the R package, using the system libarrow I compiled above. ``` (arrow) tdhock@maude-MacBookPro:~/arrow-git/cpp/build(main)$ ARROW_DEPENDENCY_SOURCE=SYSTEM ARROW_R_DEV=true LIBARROW_BINARY=false PKG_CONFIG_PATH=$HOME/lib/pkgconfig R CMD INSTALL ../../r Loading required package: grDevices * installing to library ‘/home/tdhock/lib/R/library’ * installing *source* package ‘arrow’ ... ** using staged installation *** Generating code with data-raw/codegen.R Loading required package: grDevices Error in library(decor) : there is no package called ‘decor’ Calls: suppressPackageStartupMessages -> withCallingHandlers -> library Execution halted *** Trying Arrow C++ found by pkg-config: /home/tdhock *** > Packages are both on development versions (13.0.0-SNAPSHOT, 12.0.0.9000) *** > If installation fails, rebuild the C++ library to match the R version *** > or retry with FORCE_BUNDLED_BUILD=true PKG_CFLAGS=-I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON PKG_LIBS=-L/home/tdhock/lib -larrow_acero -larrow_dataset -lparquet -larrow ** libs using C++ compiler: ‘g++ (GCC) 10.1.0’ using C++17 g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c arrowExports.cpp -o arrowExports.o g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c bridge.cpp -o bridge.o g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c buffer.cpp -o buffer.o g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c chunkedarray.cpp -o chunkedarray.o g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c compression.cpp -o compression.o g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c compute-exec.cpp -o compute-exec.o g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c compute.cpp -o compute.o g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c config.cpp -o config.o g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c csv.cpp -o csv.o g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c dataset.cpp -o dataset.o dataset.cpp: In function ‘std::shared_ptr<arrow::dataset::ParquetFileFormat> dataset___ParquetFileFormat__Make(const std::shared_ptr<arrow::dataset::ParquetFragmentScanOptions>&, cpp11::strings)’: dataset.cpp:229:6: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::ParquetFileFormat’ {aka ‘class arrow::dataset::ParquetFileFormat’} 229 | fmt->default_fragment_scan_options = std::move(options); | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:80:7: note: forward declaration of ‘using element_type = class arrow::dataset::ParquetFileFormat’ {aka ‘class arrow::dataset::ParquetFileFormat’} 80 | class ParquetFileFormat; | ^~~~~~~~~~~~~~~~~ dataset.cpp:232:16: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::ParquetFileFormat’ {aka ‘class arrow::dataset::ParquetFileFormat’} 232 | auto& d = fmt->reader_options.dict_columns; | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:80:7: note: forward declaration of ‘using element_type = class arrow::dataset::ParquetFileFormat’ {aka ‘class arrow::dataset::ParquetFileFormat’} 80 | class ParquetFileFormat; | ^~~~~~~~~~~~~~~~~ dataset.cpp: In function ‘void dataset___ParquetFileWriteOptions__update(const std::shared_ptr<arrow::dataset::ParquetFileWriteOptions>&, const std::shared_ptr<parquet::WriterProperties>&, const std::shared_ptr<parquet::ArrowWriterProperties>&)’: dataset.cpp:251:10: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::ParquetFileWriteOptions’ {aka ‘class arrow::dataset::ParquetFileWriteOptions’} 251 | options->writer_properties = writer_props; | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:84:7: note: forward declaration of ‘using element_type = class arrow::dataset::ParquetFileWriteOptions’ {aka ‘class arrow::dataset::ParquetFileWriteOptions’} 84 | class ParquetFileWriteOptions; | ^~~~~~~~~~~~~~~~~~~~~~~ dataset.cpp:252:10: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::ParquetFileWriteOptions’ {aka ‘class arrow::dataset::ParquetFileWriteOptions’} 252 | options->arrow_writer_properties = arrow_writer_props; | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:84:7: note: forward declaration of ‘using element_type = class arrow::dataset::ParquetFileWriteOptions’ {aka ‘class arrow::dataset::ParquetFileWriteOptions’} 84 | class ParquetFileWriteOptions; | ^~~~~~~~~~~~~~~~~~~~~~~ dataset.cpp: In function ‘void dataset___CsvFileWriteOptions__update(const std::shared_ptr<arrow::dataset::CsvFileWriteOptions>&, const std::shared_ptr<arrow::csv::WriteOptions>&)’: dataset.cpp:278:15: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::CsvFileWriteOptions’ {aka ‘class arrow::dataset::CsvFileWriteOptions’} 278 | *csv_options->write_options = *write_options; | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:72:7: note: forward declaration of ‘using element_type = class arrow::dataset::CsvFileWriteOptions’ {aka ‘class arrow::dataset::CsvFileWriteOptions’} 72 | class CsvFileWriteOptions; | ^~~~~~~~~~~~~~~~~~~ dataset.cpp: In function ‘std::shared_ptr<arrow::dataset::CsvFileFormat> dataset___CsvFileFormat__Make(const std::shared_ptr<arrow::csv::ParseOptions>&, const std::shared_ptr<arrow::csv::ConvertOptions>&, const std::shared_ptr<arrow::csv::ReadOptions>&)’: dataset.cpp:292:9: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::CsvFileFormat’ {aka ‘class arrow::dataset::CsvFileFormat’} 292 | format->parse_options = *parse_options; | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:70:7: note: forward declaration of ‘using element_type = class arrow::dataset::CsvFileFormat’ {aka ‘class arrow::dataset::CsvFileFormat’} 70 | class CsvFileFormat; | ^~~~~~~~~~~~~ dataset.cpp:294:36: error: invalid use of incomplete type ‘using element_type = struct arrow::dataset::CsvFragmentScanOptions’ {aka ‘struct arrow::dataset::CsvFragmentScanOptions’} 294 | if (convert_options) scan_options->convert_options = *convert_options; | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:73:8: note: forward declaration of ‘using element_type = struct arrow::dataset::CsvFragmentScanOptions’ {aka ‘struct arrow::dataset::CsvFragmentScanOptions’} 73 | struct CsvFragmentScanOptions; | ^~~~~~~~~~~~~~~~~~~~~~ dataset.cpp:295:33: error: invalid use of incomplete type ‘using element_type = struct arrow::dataset::CsvFragmentScanOptions’ {aka ‘struct arrow::dataset::CsvFragmentScanOptions’} 295 | if (read_options) scan_options->read_options = *read_options; | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:73:8: note: forward declaration of ‘using element_type = struct arrow::dataset::CsvFragmentScanOptions’ {aka ‘struct arrow::dataset::CsvFragmentScanOptions’} 73 | struct CsvFragmentScanOptions; | ^~~~~~~~~~~~~~~~~~~~~~ dataset.cpp:296:9: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::CsvFileFormat’ {aka ‘class arrow::dataset::CsvFileFormat’} 296 | format->default_fragment_scan_options = std::move(scan_options); | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:70:7: note: forward declaration of ‘using element_type = class arrow::dataset::CsvFileFormat’ {aka ‘class arrow::dataset::CsvFileFormat’} 70 | class CsvFileFormat; | ^~~~~~~~~~~~~ dataset.cpp: In function ‘std::shared_ptr<arrow::dataset::CsvFragmentScanOptions> dataset___CsvFragmentScanOptions__Make(const std::shared_ptr<arrow::csv::ConvertOptions>&, const std::shared_ptr<arrow::csv::ReadOptions>&)’: dataset.cpp:313:10: error: invalid use of incomplete type ‘using element_type = struct arrow::dataset::CsvFragmentScanOptions’ {aka ‘struct arrow::dataset::CsvFragmentScanOptions’} 313 | options->convert_options = *convert_options; | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:73:8: note: forward declaration of ‘using element_type = struct arrow::dataset::CsvFragmentScanOptions’ {aka ‘struct arrow::dataset::CsvFragmentScanOptions’} 73 | struct CsvFragmentScanOptions; | ^~~~~~~~~~~~~~~~~~~~~~ dataset.cpp:314:10: error: invalid use of incomplete type ‘using element_type = struct arrow::dataset::CsvFragmentScanOptions’ {aka ‘struct arrow::dataset::CsvFragmentScanOptions’} 314 | options->read_options = *read_options; | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:73:8: note: forward declaration of ‘using element_type = struct arrow::dataset::CsvFragmentScanOptions’ {aka ‘struct arrow::dataset::CsvFragmentScanOptions’} 73 | struct CsvFragmentScanOptions; | ^~~~~~~~~~~~~~~~~~~~~~ dataset.cpp: In function ‘std::shared_ptr<arrow::dataset::ParquetFragmentScanOptions> dataset___ParquetFragmentScanOptions__Make(bool, int64_t, bool)’: dataset.cpp:324:12: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 324 | options->reader_properties->enable_buffered_stream(); | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:82:7: note: forward declaration of ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 82 | class ParquetFragmentScanOptions; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ dataset.cpp:326:12: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 326 | options->reader_properties->disable_buffered_stream(); | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:82:7: note: forward declaration of ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 82 | class ParquetFragmentScanOptions; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ dataset.cpp:328:10: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 328 | options->reader_properties->set_buffer_size(buffer_size); | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:82:7: note: forward declaration of ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 82 | class ParquetFragmentScanOptions; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ dataset.cpp:329:10: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 329 | options->arrow_reader_properties->set_pre_buffer(pre_buffer); | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:82:7: note: forward declaration of ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 82 | class ParquetFragmentScanOptions; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ dataset.cpp:331:12: error: invalid use of incomplete type ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 331 | options->arrow_reader_properties->set_cache_options( | ^~ In file included from ./arrow_types.h:44, from dataset.cpp:18: /home/tdhock/include/arrow/dataset/type_fwd.h:82:7: note: forward declaration of ‘using element_type = class arrow::dataset::ParquetFragmentScanOptions’ {aka ‘class arrow::dataset::ParquetFragmentScanOptions’} 82 | class ParquetFragmentScanOptions; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:56, from /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:52, from /home/tdhock/include/c++/10.1.0/memory:84, from ././arrow_cpp11.h:20, from ./arrow_types.h:22, from dataset.cpp:18: /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h: In instantiation of ‘struct __gnu_cxx::__aligned_buffer<arrow::dataset::ParquetFileFormat>’: /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:538:35: required from ‘class std::_Sp_counted_ptr_inplace<arrow::dataset::ParquetFileFormat, std::allocator<arrow::dataset::ParquetFileFormat>, __gnu_cxx::_S_atomic>::_Impl’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:599:13: required from ‘class std::_Sp_counted_ptr_inplace<arrow::dataset::ParquetFileFormat, std::allocator<arrow::dataset::ParquetFileFormat>, __gnu_cxx::_S_atomic>’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:679:43: required from ‘std::__shared_count<_Lp>::__shared_count(_Tp*&, std::_Sp_alloc_shared_tag<_Alloc>, _Args&& ...) [with _Tp = arrow::dataset::ParquetFileFormat; _Alloc = std::allocator<arrow::dataset::ParquetFileFormat>; _Args = {}; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:1371:71: required from ‘std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<arrow::dataset::ParquetFileFormat>; _Args = {}; _Tp = arrow::dataset::ParquetFileFormat; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:408:59: required from ‘std::shared_ptr<_Tp>::shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<arrow::dataset::ParquetFileFormat>; _Args = {}; _Tp = arrow::dataset::ParquetFileFormat]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:859:14: required from ‘std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = arrow::dataset::ParquetFileFormat; _Alloc = std::allocator<arrow::dataset::ParquetFileFormat>; _Args = {}]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:875:39: required from ‘std::shared_ptr<_Tp> std::make_shared(_Args&& ...) [with _Tp = arrow::dataset::ParquetFileFormat; _Args = {}]’ dataset.cpp:228:54: required from here /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h:91:28: error: invalid application of ‘sizeof’ to incomplete type ‘arrow::dataset::ParquetFileFormat’ 91 | : std::aligned_storage<sizeof(_Tp), __alignof__(_Tp)> | ^~~~~~~~~~~ /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h:91:28: error: invalid application of ‘sizeof’ to incomplete type ‘arrow::dataset::ParquetFileFormat’ /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h: In instantiation of ‘struct __gnu_cxx::__aligned_buffer<arrow::dataset::CsvFileFormat>’: /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:538:35: required from ‘class std::_Sp_counted_ptr_inplace<arrow::dataset::CsvFileFormat, std::allocator<arrow::dataset::CsvFileFormat>, __gnu_cxx::_S_atomic>::_Impl’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:599:13: required from ‘class std::_Sp_counted_ptr_inplace<arrow::dataset::CsvFileFormat, std::allocator<arrow::dataset::CsvFileFormat>, __gnu_cxx::_S_atomic>’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:679:43: required from ‘std::__shared_count<_Lp>::__shared_count(_Tp*&, std::_Sp_alloc_shared_tag<_Alloc>, _Args&& ...) [with _Tp = arrow::dataset::CsvFileFormat; _Alloc = std::allocator<arrow::dataset::CsvFileFormat>; _Args = {}; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:1371:71: required from ‘std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<arrow::dataset::CsvFileFormat>; _Args = {}; _Tp = arrow::dataset::CsvFileFormat; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:408:59: required from ‘std::shared_ptr<_Tp>::shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<arrow::dataset::CsvFileFormat>; _Args = {}; _Tp = arrow::dataset::CsvFileFormat]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:859:14: required from ‘std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = arrow::dataset::CsvFileFormat; _Alloc = std::allocator<arrow::dataset::CsvFileFormat>; _Args = {}]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:875:39: required from ‘std::shared_ptr<_Tp> std::make_shared(_Args&& ...) [with _Tp = arrow::dataset::CsvFileFormat; _Args = {}]’ dataset.cpp:291:53: required from here /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h:91:28: error: invalid application of ‘sizeof’ to incomplete type ‘arrow::dataset::CsvFileFormat’ /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h:91:28: error: invalid application of ‘sizeof’ to incomplete type ‘arrow::dataset::CsvFileFormat’ /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h: In instantiation of ‘struct __gnu_cxx::__aligned_buffer<arrow::dataset::CsvFragmentScanOptions>’: /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:538:35: required from ‘class std::_Sp_counted_ptr_inplace<arrow::dataset::CsvFragmentScanOptions, std::allocator<arrow::dataset::CsvFragmentScanOptions>, __gnu_cxx::_S_atomic>::_Impl’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:599:13: required from ‘class std::_Sp_counted_ptr_inplace<arrow::dataset::CsvFragmentScanOptions, std::allocator<arrow::dataset::CsvFragmentScanOptions>, __gnu_cxx::_S_atomic>’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:679:43: required from ‘std::__shared_count<_Lp>::__shared_count(_Tp*&, std::_Sp_alloc_shared_tag<_Alloc>, _Args&& ...) [with _Tp = arrow::dataset::CsvFragmentScanOptions; _Alloc = std::allocator<arrow::dataset::CsvFragmentScanOptions>; _Args = {}; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:1371:71: required from ‘std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<arrow::dataset::CsvFragmentScanOptions>; _Args = {}; _Tp = arrow::dataset::CsvFragmentScanOptions; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:408:59: required from ‘std::shared_ptr<_Tp>::shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<arrow::dataset::CsvFragmentScanOptions>; _Args = {}; _Tp = arrow::dataset::CsvFragmentScanOptions]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:859:14: required from ‘std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = arrow::dataset::CsvFragmentScanOptions; _Alloc = std::allocator<arrow::dataset::CsvFragmentScanOptions>; _Args = {}]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:875:39: required from ‘std::shared_ptr<_Tp> std::make_shared(_Args&& ...) [with _Tp = arrow::dataset::CsvFragmentScanOptions; _Args = {}]’ dataset.cpp:293:68: required from here /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h:91:28: error: invalid application of ‘sizeof’ to incomplete type ‘arrow::dataset::CsvFragmentScanOptions’ /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h:91:28: error: invalid application of ‘sizeof’ to incomplete type ‘arrow::dataset::CsvFragmentScanOptions’ /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h: In instantiation of ‘struct __gnu_cxx::__aligned_buffer<arrow::dataset::ParquetFragmentScanOptions>’: /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:538:35: required from ‘class std::_Sp_counted_ptr_inplace<arrow::dataset::ParquetFragmentScanOptions, std::allocator<arrow::dataset::ParquetFragmentScanOptions>, __gnu_cxx::_S_atomic>::_Impl’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:599:13: required from ‘class std::_Sp_counted_ptr_inplace<arrow::dataset::ParquetFragmentScanOptions, std::allocator<arrow::dataset::ParquetFragmentScanOptions>, __gnu_cxx::_S_atomic>’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:679:43: required from ‘std::__shared_count<_Lp>::__shared_count(_Tp*&, std::_Sp_alloc_shared_tag<_Alloc>, _Args&& ...) [with _Tp = arrow::dataset::ParquetFragmentScanOptions; _Alloc = std::allocator<arrow::dataset::ParquetFragmentScanOptions>; _Args = {}; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr_base.h:1371:71: required from ‘std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<arrow::dataset::ParquetFragmentScanOptions>; _Args = {}; _Tp = arrow::dataset::ParquetFragmentScanOptions; __gnu_cxx::_Lock_policy _Lp = __gnu_cxx::_S_atomic]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:408:59: required from ‘std::shared_ptr<_Tp>::shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator<arrow::dataset::ParquetFragmentScanOptions>; _Args = {}; _Tp = arrow::dataset::ParquetFragmentScanOptions]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:859:14: required from ‘std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = arrow::dataset::ParquetFragmentScanOptions; _Alloc = std::allocator<arrow::dataset::ParquetFragmentScanOptions>; _Args = {}]’ /home/tdhock/include/c++/10.1.0/bits/shared_ptr.h:875:39: required from ‘std::shared_ptr<_Tp> std::make_shared(_Args&& ...) [with _Tp = arrow::dataset::ParquetFragmentScanOptions; _Args = {}]’ dataset.cpp:322:67: required from here /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h:91:28: error: invalid application of ‘sizeof’ to incomplete type ‘arrow::dataset::ParquetFragmentScanOptions’ /home/tdhock/include/c++/10.1.0/ext/aligned_buffer.h:91:28: error: invalid application of ‘sizeof’ to incomplete type ‘arrow::dataset::ParquetFragmentScanOptions’ /home/tdhock/lib/R/etc/Makeconf:198: recipe for target 'dataset.o' failed make: *** [dataset.o] Error 1 ERROR: compilation failed for package ‘arrow’ * removing ‘/home/tdhock/lib/R/library/arrow’ * restoring previous ‘/home/tdhock/lib/R/library/arrow’ ``` The output above indicates that there is a missing include. Notice that the g++ command that failed, g++ -std=gnu++17 -I"/home/tdhock/lib/R/include" -DNDEBUG -I/home/tdhock/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -I'/home/tdhock/lib/R/library/cpp11/include' -I/usr/local/include -fpic -g -O2 -c dataset.cpp -o dataset.o, is missing -DARROW_PARQUET -DARROW_CSV. If those two definitions are present then this command line works fine, because the the right header files are included from here, https://github.com/apache/arrow/blob/14f9bf923a5655c04130a2db0cabe15a0249dd79/cpp/src/arrow/dataset/api.h The above suggests there is some missing logic in the header files. Something like: if ARROW_R_WITH_PARQUET is defined, then ARROW_PARQUET should be defined as well, so we can get those header files when we need them, etc. ### Component(s) R -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
