[
https://issues.apache.org/jira/browse/ARROW-16680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570394#comment-17570394
]
Vitalie Spinu commented on ARROW-16680:
---------------------------------------
Some more digging with more debug symbols reveals that the error happens in the
finalizer.
I am a complete nob around these topics but it looks to me that at some point
the finalizer is triggered and when `Aws::S3::S3Client::S3Client` is destructed
curl still attempts to write to a disconnected socket in `curl_easy_cleanup`.
{code:java}
Thread 1 "R" received signal SIGPIPE, Broken pipe.
__libc_write (nbytes=31, buf=0x555576a55b83, fd=43) at
../sysdeps/unix/sysv/linux/write.c:26
26 ../sysdeps/unix/sysv/linux/write.c: No such file or directory.
(gdb) break
break break-range
(gdb) backtrace
#0 __libc_write (nbytes=31, buf=0x555576a55b83, fd=43) at
../sysdeps/unix/sysv/linux/write.c:26
#1 __libc_write (fd=43, buf=0x555576a55b83, nbytes=31) at
../sysdeps/unix/sysv/linux/write.c:24
#2 0x00007fffe92b1459 in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#3 0x00007fffe92ac63e in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#4 0x00007fffe92ab654 in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#5 0x00007fffe92abb17 in BIO_write () from
/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
#6 0x00007fffe90f7dde in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
#7 0x00007fffe90f8cd9 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
#8 0x00007fffe910288e in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
#9 0x00007fffe9100a65 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1
#10 0x00007fffe910bec3 in SSL_shutdown () from
/usr/lib/x86_64-linux-gnu/libssl.so.1.1
#11 0x00007fffe91cc7c5 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#12 0x00007fffe91cc835 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#13 0x00007fffe918a8ce in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#14 0x00007fffe91b795b in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#15 0x00007fffe919f336 in curl_multi_cleanup () from
/usr/lib/x86_64-linux-gnu/libcurl.so.4
#16 0x00007fffe918ab43 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so.4
#17 0x00007fffe91972ed in curl_easy_cleanup () from
/usr/lib/x86_64-linux-gnu/libcurl.so.4
#18 0x00007fffec96aa14 in Aws::Http::CurlHandleContainer::~CurlHandleContainer
(this=0x55556bd25ff8, __in_chrg=<optimized out>)
at
~/vc/arrow/cpp/build/awssdk_ep-prefix/src/awssdk_ep/aws-cpp-sdk-core/source/http/curl/CurlHandleContainer.cpp:31
#19 0x00007fffec952377 in Aws::Http::CurlHttpClient::~CurlHttpClient
(this=0x55556bd25f90, __in_chrg=<optimized out>) at
/usr/include/c++/9/ext/new_allocator.h:89
#20 0x00007fffeabde0f2 in
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
(this=0x55556bd25f80) at /usr/include/c++/9/bits/shared_ptr_base.h:155
#21 0x00007fffec7f4ba5 in
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count
(this=0x5555758352d0, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr_base.h:1169
#22 std::__shared_ptr<Aws::Http::HttpClient,
(__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0x5555758352c8,
__in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr_base.h:1169
#23 std::shared_ptr<Aws::Http::HttpClient>::~shared_ptr (this=0x5555758352c8,
__in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr.h:103
#24 Aws::Client::AWSClient::~AWSClient (this=0x5555758352a0,
__in_chrg=<optimized out>) at
~/vc/arrow/cpp/build/awssdk_ep-prefix/src/awssdk_ep/aws-cpp-sdk-core/include/aws/core/client/AWSClient.h:106
#25 Aws::Client::AWSXMLClient::~AWSXMLClient (this=0x5555758352a0,
__in_chrg=<optimized out>) at
~/vc/arrow/cpp/build/awssdk_ep-prefix/src/awssdk_ep/aws-cpp-sdk-core/include/aws/core/client/AWSClient.h:401
#26 Aws::S3::S3Client::~S3Client (this=0x5555758352a0, __in_chrg=<optimized
out>) at
~/vc/arrow/cpp/build/awssdk_ep-prefix/src/awssdk_ep/aws-cpp-sdk-s3/source/S3Client.cpp:160
#27 0x00007fffec52daf4 in arrow::fs::(anonymous namespace)::S3Client::~S3Client
(this=0x5555758352a0, __in_chrg=<optimized out>) at
~/vc/arrow/cpp/src/arrow/filesystem/s3fs.cc:554
#28 0x00007fffec52eefd in __gnu_cxx::new_allocator<arrow::fs::(anonymous
namespace)::S3Client>::destroy<arrow::fs::(anonymous namespace)::S3Client>
(this=0x5555758352a0, __p=0x5555758352a0)
at /usr/include/c++/9/ext/new_allocator.h:152
#29 0x00007fffec52eb6f in
std::allocator_traits<std::allocator<arrow::fs::(anonymous
namespace)::S3Client> >::destroy<arrow::fs::(anonymous namespace)::S3Client>
(__a=..., __p=0x5555758352a0)
at /usr/include/c++/9/bits/alloc_traits.h:496
#30 0x00007fffec52e4e1 in std::_Sp_counted_ptr_inplace<arrow::fs::(anonymous
namespace)::S3Client, std::allocator<arrow::fs::(anonymous
namespace)::S3Client>, (__gnu_cxx::_Lock_policy)2>::_M_dispose
(this=0x555575835290)
at /usr/include/c++/9/bits/shared_ptr_base.h:557
#31 0x00007fffeabde0f2 in
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
(this=0x555575835290) at /usr/include/c++/9/bits/shared_ptr_base.h:155
#32 0x00007fffeabda9bb in
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count
(this=0x5555685ba510, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr_base.h:730
#33 0x00007fffec511cc4 in std::__shared_ptr<arrow::fs::(anonymous
namespace)::S3Client, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr
(this=0x5555685ba508, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr_base.h:1169
#34 0x00007fffec511ce0 in std::shared_ptr<arrow::fs::(anonymous
namespace)::S3Client>::~shared_ptr (this=0x5555685ba508, __in_chrg=<optimized
out>) at /usr/include/c++/9/bits/shared_ptr.h:103
#35 0x00007fffec57c38e in arrow::fs::S3FileSystem::Impl::~Impl
(this=0x5555685ba0d0, __in_chrg=<optimized out>) at
~/vc/arrow/cpp/src/arrow/filesystem/s3fs.cc:1647
#36 0x00007fffec57c3e0 in
__gnu_cxx::new_allocator<arrow::fs::S3FileSystem::Impl>::destroy<arrow::fs::S3FileSystem::Impl>
(this=0x5555685ba0d0, __p=0x5555685ba0d0) at
/usr/include/c++/9/ext/new_allocator.h:152
#37 0x00007fffec57aff5 in
std::allocator_traits<std::allocator<arrow::fs::S3FileSystem::Impl>
>::destroy<arrow::fs::S3FileSystem::Impl> (__a=..., __p=0x5555685ba0d0) at
/usr/include/c++/9/bits/alloc_traits.h:496
#38 0x00007fffec57995f in
std::_Sp_counted_ptr_inplace<arrow::fs::S3FileSystem::Impl,
std::allocator<arrow::fs::S3FileSystem::Impl>,
(__gnu_cxx::_Lock_policy)2>::_M_dispose (this=0x5555685ba0c0)
at /usr/include/c++/9/bits/shared_ptr_base.h:557
#39 0x00007fffeabde0f2 in
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
(this=0x5555685ba0c0) at /usr/include/c++/9/bits/shared_ptr_base.h:155
#40 0x00007fffeabda9bb in
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count
(this=0x555568408930, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr_base.h:730
#41 0x00007fffec536ec6 in std::__shared_ptr<arrow::fs::S3FileSystem::Impl,
(__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0x555568408928,
__in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr_base.h:1169
#42 0x00007fffec536ee6 in
std::shared_ptr<arrow::fs::S3FileSystem::Impl>::~shared_ptr
(this=0x555568408928, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr.h:103
#43 0x00007fffec51b3da in arrow::fs::S3FileSystem::~S3FileSystem
(this=0x5555684088e0, __in_chrg=<optimized out>) at
~/vc/arrow/cpp/src/arrow/filesystem/s3fs.cc:2206
#44 0x00007fffec51b406 in arrow::fs::S3FileSystem::~S3FileSystem
(this=0x5555684088e0, __in_chrg=<optimized out>) at
~/vc/arrow/cpp/src/arrow/filesystem/s3fs.cc:2206
#45 0x00007fffec57ab32 in std::_Sp_counted_ptr<arrow::fs::S3FileSystem*,
(__gnu_cxx::_Lock_policy)2>::_M_dispose (this=0x555565dc4cb0) at
/usr/include/c++/9/bits/shared_ptr_base.h:377
#46 0x00007fffee754ed8 in
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
(this=0x555565dc4cb0) at /usr/include/c++/9/bits/shared_ptr_base.h:148
#47 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
(this=0x555565dc4cb0) at /usr/include/c++/9/bits/shared_ptr_base.h:148
#48 0x00007fffee35eba9 in
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count
(this=0x555570637880, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr_base.h:730
#49 0x00007fffee37b412 in std::__shared_ptr<arrow::fs::FileSystem,
(__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0x555570637878,
__in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr_base.h:1169
#50 0x00007fffee37b45c in std::shared_ptr<arrow::fs::FileSystem>::~shared_ptr
(this=0x555570637878, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr.h:103
#51 0x00007fffec49b75a in arrow::fs::SubTreeFileSystem::~SubTreeFileSystem
(this=0x555570637810, __in_chrg=<optimized out>) at
~/vc/arrow/cpp/src/arrow/filesystem/filesystem.cc:276
#52 0x00007fffee754ed8 in
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
(this=0x555570637800) at /usr/include/c++/9/bits/shared_ptr_base.h:148
#53 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
(this=0x555570637800) at /usr/include/c++/9/bits/shared_ptr_base.h:148
#54 0x00007fffee35eba9 in
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count
(this=0x555568218770, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr_base.h:730
#55 0x00007fffee37b412 in std::__shared_ptr<arrow::fs::FileSystem,
(__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0x555568218768,
__in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr_base.h:1169
#56 0x00007fffee37b45c in std::shared_ptr<arrow::fs::FileSystem>::~shared_ptr
(this=0x555568218768, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr.h:103
#57 0x00007fffee3c23d4 in arrow::dataset::FileSystemDataset::~FileSystemDataset
(this=0x555568218720, __in_chrg=<optimized out>) at
~/vc/arrow/cpp/src/arrow/dataset/file_base.h:222
#58 0x00007fffee3c2410 in arrow::dataset::FileSystemDataset::~FileSystemDataset
(this=0x555568218720, __in_chrg=<optimized out>) at
~/vc/arrow/cpp/src/arrow/dataset/file_base.h:222
#59 0x00007fffee3d19e2 in
std::_Sp_counted_ptr<arrow::dataset::FileSystemDataset*,
(__gnu_cxx::_Lock_policy)2>::_M_dispose (this=0x55556a3ca000) at
/usr/include/c++/9/bits/shared_ptr_base.h:377
#60 0x00007fffee7ad357 in
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
(this=0x55556a3ca000) at /usr/include/c++/9/bits/shared_ptr_base.h:148
#61 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
(this=0x55556a3ca000) at /usr/include/c++/9/bits/shared_ptr_base.h:148
#62 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count
(this=0x555568170738, __in_chrg=<optimized out>) at
/usr/include/c++/9/bits/shared_ptr_base.h:730
#63 std::__shared_ptr<arrow::dataset::Dataset,
(__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0x555568170730,
__in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr_base.h:1169
#64 std::shared_ptr<arrow::dataset::Dataset>::~shared_ptr (this=0x555568170730,
__in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr.h:103
#65 cpp11::default_deleter<std::shared_ptr<arrow::dataset::Dataset> >
(obj=0x555568170730) at
/path/to/renv/library/R-4.2/x86_64-pc-linux-gnu/cpp11/include/cpp11/external_pointer.hpp:17
#66 cpp11::external_pointer<std::shared_ptr<arrow::dataset::Dataset>,
&cpp11::default_deleter<std::shared_ptr<arrow::dataset::Dataset> > >::r_deleter
(p=<optimized out>) at
/renv/library/R-4.2/x86_64-pc-linux-gnu/cpp11/include/cpp11/external_pointer.hpp:47
#67 cpp11::external_pointer<std::shared_ptr<arrow::dataset::Dataset>,
&cpp11::default_deleter<std::shared_ptr<arrow::dataset::Dataset> > >::r_deleter
(p=<optimized out>) at
/renv/library/R-4.2/x86_64-pc-linux-gnu/cpp11/include/cpp11/external_pointer.hpp:36
#68 0x00005555556dcb77 in R_RunWeakRefFinalizer (w=<optimized out>) at
memory.c:1500
#69 0x00005555556dcde3 in RunFinalizers () at memory.c:1567
#70 0x00005555556dcf85 in R_RunPendingFinalizers () at memory.c:1603
#71 0x00005555556937ba in bc_check_sigint () at eval.c:5550
#72 bcEval (body=<optimized out>, rho=<optimized out>, useCache=<optimized
out>) at eval.c:6744
#73 0x000055555569cd58 in Rf_eval (e=0x5555678ed528,
rho=rho@entry=0x55557677c728) at eval.c:748
#74 0x000055555569ea8f in R_execClosure (call=call@entry=0x555567b579f8,
newrho=newrho@entry=0x55557677c728, sysparent=<optimized out>,
rho=rho@entry=0x5555767b7008, arglist=arglist@entry=0x55557677c6b8,
op=op@entry=0x5555678ed1a8) at eval.c:1918
{code}
> [R] Weird R error: Error in
> fs___FileSystem__GetTargetInfos_FileSelector(self, x) : ignoring SIGPIPE
> signal
> --------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-16680
> URL: https://issues.apache.org/jira/browse/ARROW-16680
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
> Affects Versions: 8.0.0
> Reporter: Carl Boettiger
> Priority: Major
>
> Okay apologies, this is a bit of a weird error but is annoying the heck out
> of me. The following block of all R code, when run with Rscript (or embedded
> into any form of Rmd, quarto, knitr doc) produces the error below (at least
> most of the time):
>
> {code:java}
> library(arrow)
> library(dplyr){code}
> {code:java}
> Sys.setenv(AWS_EC2_METADATA_DISABLED = "TRUE")
> Sys.unsetenv("AWS_ACCESS_KEY_ID")
> Sys.unsetenv("AWS_SECRET_ACCESS_KEY")
> Sys.unsetenv("AWS_DEFAULT_REGION")
> Sys.unsetenv("AWS_S3_ENDPOINT")s3 <- arrow::s3_bucket(bucket =
> "scores/parquet",
> endpoint_override = "data.ecoforecast.org")
> ds <- arrow::open_dataset(s3, partitioning = c("theme", "year"))
> ds |> dplyr::filter(theme == "phenology") |> dplyr::collect()
> {code}
> Gives the error
>
>
> {code:java}
> Error in fs___FileSystem__GetTargetInfos_FileSelector(self, x) :
> ignoring SIGPIPE signal
> Calls: %>% ... <Anonymous> -> fs___FileSystem__GetTargetInfos_FileSelector
> {code}
> But only when run as a script! When run interactively in an R console, this
> code runs just fine. Even as a script the code seems to run fine, but
> erroneously seems to be attempting this sigpipe I don't understand.
> If the script is executed with litter
> ([https://dirk.eddelbuettel.com/code/littler.html)] then it runs fine, since
> littler handles sigpipe but Rscripts don't. But I have no idea why the above
> code throws a pipe in the first place. Worse, if I choose a different filter
> for the above, like "aquatics", it (usually) works without the error.
> I have no idea why `fs___FileSystem__GetTargetInfos_FileSelector` results in
> this, but would really appreciate any hints on how to avoid this as it makes
> it very hard to use arrow in workflows right now!
>
> thanks for all you do!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
