jmahlik opened a new issue, #36980: URL: https://github.com/apache/arrow/issues/36980
### Describe the bug, including details regarding any error messages, version, and platform. # Description of the bug Terminate called without an active exception (core dumped) when writing datasets just before python finalizes. I could not find a way to shut down the active threads on python interpreter finalization. Is there a way to tell the thread pool to shut down? Any thoughts on potential workarounds appreciated :). This may be related to following but thought I would open it since it's a bit different. It's pretty common to write output at the end of a script/program. * https://github.com/apache/arrow/issues/31486 * https://github.com/apache/arrow/issues/34314 # Repro This doesn't reproduce every time, it usually segfaults about 1/25 or so times. `repro.py`: Writing a dataset as parquet ```python import tempfile import string import numpy as np import pandas as pd import pyarrow as pa import pyarrow.dataset as ds NUM_ROWS = 10_000 def file_visitor(written_file): print(f"path={written_file.path}") print(f"size={written_file.size} bytes") print(f"metadata={written_file.metadata}") def main() -> int: df = pd.DataFrame( { "float": np.random.rand(NUM_ROWS), "int": np.random.randint(0, 10000, size=NUM_ROWS), "string": np.random.choice(list(string.ascii_letters), size=NUM_ROWS), } ) data = pa.Table.from_pandas( df, ) print(pa.show_info()) with tempfile.TemporaryDirectory() as t: ds.write_dataset( data=data, format="parquet", base_dir=t, max_rows_per_file=1000, max_rows_per_group=1000, # So we can see all the files being written file_visitor=file_visitor, use_threads=True, ) print(f"Wrote data to {t}") return 0 if __name__ == "__main__": # Doesn't matter if you do exit(main()) or just main(), same thing happens raise SystemExit(main()) ``` `test_segfault.py`: It takes about 20-30 tries for me to repro. So this runs the repro a bunch. ```python import subprocess import sys NUM_RUNS = 1_000 commands = [(sys.executable, "repro.py") for _ in range(NUM_RUNS)] n = 5 for j in range(max(int(len(commands) / n), 1)): procs = [ subprocess.Popen(i, stdout=subprocess.PIPE, stderr=subprocess.PIPE) for i in commands[j * n : min((j + 1) * n, len(commands))] ] print(f"Running {j}-{j+n}") for p in procs: stdout, stderr = p.communicate() print(stdout.decode()) print(stderr.decode()) if p.returncode != 0: print(p.returncode) raise ValueError(f"Core dump {stderr.decode()}") ``` # Dockerfile ```docker FROM python:3.11.4-slim-buster RUN pip install pyarrow pandas -U COPY test_segfault.py /opt/segfault/ COPY repro.py /opt/segfault/ ``` ## Running it: Attempt to set ulimit=unlimited and mount the pwd so that we can maybe capture a core file. I wasn't able to since my host is windows, but we were able to get one from a linux host in production. ```bash docker run --rm -it --ulimit core=-1 -v /$(PWD)/segfault/:/opt/segfault bash -c 'cd /opt/segfault && python ./test_segfault.py' ``` # Versions/Platform I can only repro this in docker on a debian image. Not able to get it to trigger on windows. ``` pyarrow version info -------------------- Package kind : python-wheel-manylinux2014 Arrow C++ library version : 12.0.0 Arrow C++ compiler : GNU 10.2.1 Arrow C++ compiler flags : -fdiagnostics-color=always Arrow C++ git revision : Arrow C++ git description : Arrow C++ build type : release Platform: OS / Arch : Linux x86_64 SIMD Level : avx2 Detected SIMD Level : avx2 Memory: Default backend : jemalloc Bytes allocated : 50112 bytes Max memory : 59008 bytes Supported Backends : jemalloc, mimalloc, system Optional modules: csv : Enabled cuda : - dataset : Enabled feather : Enabled flight : Enabled fs : Enabled gandiva : - json : Enabled orc : Enabled parquet : Enabled Filesystems: GcsFileSystem : Enabled HadoopFileSystem : Enabled S3FileSystem : Enabled Compression Codecs: brotli : Enabled bz2 : Enabled gzip : Enabled lz4_frame : Enabled lz4 : Enabled snappy : Enabled zstd : Enabled ``` # Full Backtrace I can't share the entire core file since it's from a prod environment. Here's the relevant parts. <details> ```console #22 58.13 Using executable found in the core file: /opt/.venv/bin/python #22 58.13 INFO(process_core): Using library search path: #22 58.30 #22 58.30 Core file information: #22 58.30 state: D zombie: True niceness: 0 #22 58.30 pid: 17 ppid: 1 sid: 1 #22 58.30 uid: 0 gid: 0 pgrp: 1 #22 58.30 #22 58.31 The process died due receiving signal SIGABRT #22 58.37 INFO(process_core): Analyzing core file /opt/sm-scripts/core.17 with executable /opt/.venv/bin/python using stack method StackMethod.ALL with native mode NativeReportingMode.ALL #22 58.73 INFO(parse_maps_file_for_binary): python binary first map found: VirtualMap(start=0x0000563006737000, end=0x0000563006738000, filesize=0x1000, offset=0x33000, device='', flags='r', inode=0, path='/opt/.venv/bin/python') #22 58.73 INFO(parse_maps_file_for_binary): 'libpython3.11.so.1.0' first map found: VirtualMap(start=0x00007f7ac76a9000, end=0x00007f7ac7797000, filesize=0x1000, offset=0x11e18d000, device='', flags='r', inode=0, path='/usr/local/bin/../lib/libpython3.11.so.1.0') #22 58.73 INFO(_get_bss): Determined exact addr of .bss section: 0x7f7ac7b926c0 (0x7f7ac76a9000 + 0x4e96c0) #22 58.73 INFO(parse_maps_file_for_binary): bss map found: VirtualMap(start=0x00007f7ac7b926c0, end=0x00007f7ac7bd4ed8, filesize=0x42818, offset=0x11e2ed6c0, device='', flags='', inode=0, path='None') #22 58.85 INFO(process_core): Attempting to find symbol 'Py_Version' in libpython3.11.so.1.0 #22 58.85 INFO(process_core): Symbol 'Py_Version' found at address 0x7f7ac79adca8 #22 58.85 INFO(process_core): Attempting to find symbol 'Py_Version' in python #22 58.86 INFO(process_core): Trying to resolve PyInterpreterState from Elf data #22 58.86 INFO(process_core): Searching for PyInterpreterState based on PyRuntime address 0x7f7ac7b69ba0 #22 58.86 INFO(process_core): Address of PyInterpreterState found using ELF data at 0x7F7AC7B781D8 #22 58.86 INFO(process_core): Fetching Python threads #22 58.87 INFO(process_core): Attempting to find symbol '_PyRuntime' in libpython3.11.so.1.0 #22 58.87 INFO(process_core): Symbol '_PyRuntime' found at address 0x7f7ac7b69ba0 #22 58.87 INFO(process_core): Attempting to find symbol '_PyRuntime' in python #22 58.87 INFO(process_core): Constructing new Python thread with tid 17 #22 58.91 Traceback for thread 17 [Has the GIL] (most recent call last): #22 58.91 (C) File "???", line 0, in _start (python) #22 58.91 (C) File "???", line 0, in __libc_start_main (libc.so.6) #22 58.91 (C) File "???", line 0, in Py_BytesMain (libpython3.11.so.1.0) #22 58.91 (C) File "???", line 0, in Py_RunMain (libpython3.11.so.1.0) #22 58.91 (C) File "???", line 0, in Py_Exit (libpython3.11.so.1.0) #22 58.91 (C) File "???", line 0, in Py_FinalizeEx (libpython3.11.so.1.0) #22 58.91 (C) File "???", line 0, in PyGC_Collect (libpython3.11.so.1.0) #22 58.91 #22 58.91 INFO(process_core): Fetching native threads #22 59.63 INFO(process_core): Constructing new native thread with tid 36 #22 59.65 INFO(process_core): Constructing new native thread with tid 21 #22 59.65 INFO(process_core): Constructing new native thread with tid 31 #22 59.66 INFO(process_core): Constructing new native thread with tid 20 #22 59.66 INFO(process_core): Constructing new native thread with tid 18 #22 59.66 INFO(process_core): Constructing new native thread with tid 19 #22 59.67 INFO(process_core): Constructing new native thread with tid 22 #22 59.67 INFO(process_core): Constructing new native thread with tid 23 #22 59.68 INFO(process_core): Constructing new native thread with tid 26 #22 59.68 INFO(process_core): Constructing new native thread with tid 24 #22 59.69 INFO(process_core): Constructing new native thread with tid 25 #22 59.69 INFO(process_core): Constructing new native thread with tid 27 #22 59.70 INFO(process_core): Constructing new native thread with tid 32 #22 59.70 INFO(process_core): Constructing new native thread with tid 28 #22 59.75 INFO(process_core): Constructing new native thread with tid 38 #22 59.75 INFO(process_core): Constructing new native thread with tid 29 #22 59.76 INFO(process_core): Constructing new native thread with tid 40 #22 59.76 INFO(process_core): Constructing new native thread with tid 53 #22 59.76 INFO(process_core): Constructing new native thread with tid 57 #22 59.76 INFO(process_core): Constructing new native thread with tid 49 #22 59.76 INFO(process_core): Constructing new native thread with tid 39 #22 59.76 INFO(process_core): Constructing new native thread with tid 50 #22 59.77 INFO(process_core): Constructing new native thread with tid 30 #22 59.77 INFO(process_core): Constructing new native thread with tid 47 #22 59.77 INFO(process_core): Constructing new native thread with tid 43 #22 59.77 INFO(process_core): Constructing new native thread with tid 54 #22 59.77 INFO(process_core): Constructing new native thread with tid 56 #22 59.78 INFO(process_core): Constructing new native thread with tid 48 #22 59.78 INFO(process_core): Constructing new native thread with tid 44 #22 59.78 INFO(process_core): Constructing new native thread with tid 51 #22 59.84 INFO(process_core): Constructing new native thread with tid 41 #22 59.84 INFO(process_core): Constructing new native thread with tid 45 #22 59.84 INFO(process_core): Constructing new native thread with tid 42 #22 59.84 INFO(process_core): Constructing new native thread with tid 55 #22 59.90 INFO(process_core): Constructing new native thread with tid 33 #22 59.90 INFO(process_core): Constructing new native thread with tid 58 #22 59.90 INFO(process_core): Constructing new native thread with tid 52 #22 59.90 INFO(process_core): Constructing new native thread with tid 46 #22 59.91 INFO(process_core): Constructing new native thread with tid 37 #22 59.91 Traceback for thread 36 [] (most recent call last): #22 59.91 (C) File "???", line 0, in __clone (libc.so.6) #22 59.91 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.91 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void ()>::operator()() && (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void ()>::FnImpl<std::_Bind<arrow::detail::ContinueFuture (arrow::Future<arrow::internal::Empty>, arrow::io::FileInterface::CloseAsync()::{lambda()#1})> >::invoke() (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in void arrow::Future<arrow::internal::Empty>::MarkFinished<arrow::internal::Empty, void>(arrow::Status) (libarrow_python.so) #22 59.91 (C) File "???", line 0, in arrow::ConcreteFutureImpl::DoMarkFinishedOrFailed(arrow::FutureState) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void (arrow::FutureImpl const&)>::FnImpl<arrow::Future<arrow::internal::Empty>::WrapStatusyOnComplete::Callback<arrow::detail::MarkNextFinished<arrow::Future<arrow::internal::Empty>, arrow::Future<arrow::internal::Empty>, true, true> > >::invoke(arrow::FutureImpl const&) (libarrow_dataset.so.1100) #22 59.91 (C) File "???", line 0, in void arrow::Future<arrow::internal::Empty>::MarkFinished<arrow::internal::Empty, void>(arrow::Status) (libarrow_python.so) #22 59.91 (C) File "???", line 0, in arrow::ConcreteFutureImpl::DoMarkFinishedOrFailed(arrow::FutureState) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void (arrow::FutureImpl const&)>::FnImpl<arrow::Future<arrow::internal::Empty>::WrapResultyOnComplete::Callback<arrow::Future<arrow::internal::Empty>::ThenOnComplete<arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::DoFinish()::{lambda()#1}, arrow::Future<arrow::internal::Empty>::PassthruOnFailure<{lambda()#1}> > > >::invoke(arrow::FutureImpl const&) (libarrow_dataset.so.1100) #22 59.91 (C) File "???", line 0, in void arrow::Future<arrow::internal::Empty>::MarkFinished<arrow::internal::Empty, void>(arrow::Status) (libarrow_python.so) #22 59.91 (C) File "???", line 0, in arrow::ConcreteFutureImpl::DoMarkFinishedOrFailed(arrow::FutureState) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void (arrow::FutureImpl const&)>::FnImpl<arrow::Future<arrow::internal::Empty>::WrapResultyOnComplete::Callback<arrow::Future<arrow::internal::Empty>::ThenOnComplete<arrow::util::(anonymous namespace)::AsyncTaskGroupImpl::AddTask(std::unique_ptr<arrow::util::AsyncTaskScheduler::Task, std::default_delete<arrow::util::AsyncTaskScheduler::Task> >)::WrapperTask::operator()()::{lambda()#1}, arrow::Future<arrow::internal::Empty>::PassthruOnFailure<{lambda()#1}> > > >::invoke(arrow::FutureImpl const&) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in void arrow::Future<arrow::internal::Empty>::MarkFinished<arrow::internal::Empty, void>(arrow::Status) (libarrow_python.so) #22 59.91 (C) File "???", line 0, in arrow::ConcreteFutureImpl::DoMarkFinishedOrFailed(arrow::FutureState) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void (arrow::FutureImpl const&)>::FnImpl<arrow::Future<arrow::internal::Empty>::WrapResultyOnComplete::Callback<arrow::Future<arrow::internal::Empty>::ThenOnComplete<arrow::util::(anonymous namespace)::ThrottledAsyncTaskSchedulerImpl::SubmitTask(std::unique_ptr<arrow::util::AsyncTaskScheduler::Task, std::default_delete<arrow::util::AsyncTaskScheduler::Task> >, int)::{lambda()#1}::operator()()::{lambda()#1}, arrow::Future<arrow::internal::Empty>::PassthruOnFailure<{lambda()#1}> > > >::invoke(arrow::FutureImpl const&) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in void arrow::Future<arrow::internal::Empty>::MarkFinished<arrow::internal::Empty, void>(arrow::Status) (libarrow_python.so) #22 59.91 (C) File "???", line 0, in arrow::ConcreteFutureImpl::DoMarkFinishedOrFailed(arrow::FutureState) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void (arrow::FutureImpl const&)>::FnImpl<arrow::Future<arrow::internal::Empty>::WrapStatusyOnComplete::Callback<arrow::util::(anonymous namespace)::AsyncTaskSchedulerImpl::DoSubmitTask(std::unique_ptr<arrow::util::AsyncTaskScheduler::Task, std::default_delete<arrow::util::AsyncTaskScheduler::Task> >)::{lambda()#1}::operator()()::{lambda(arrow::Status const&)#1}> >::invoke(arrow::FutureImpl const&) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in void arrow::Future<arrow::internal::Empty>::MarkFinished<arrow::internal::Empty, void>(arrow::Status) (libarrow_python.so) #22 59.91 (C) File "???", line 0, in arrow::ConcreteFutureImpl::DoMarkFinishedOrFailed(arrow::FutureState) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void (arrow::FutureImpl const&)>::FnImpl<arrow::Future<arrow::internal::Empty>::WrapResultyOnComplete::Callback<arrow::Future<arrow::internal::Empty>::ThenOnComplete<arrow::util::AsyncTaskScheduler::Make(arrow::internal::FnOnce<arrow::Status (arrow::util::AsyncTaskScheduler*)>, arrow::internal::FnOnce<void (arrow::Status const&)>, arrow::StopToken)::{lambda()#1}, arrow::Future<arrow::internal::Empty>::PassthruOnFailure<{lambda()#1}> > > >::invoke(arrow::FutureImpl const&) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in void arrow::Future<arrow::internal::Empty>::MarkFinished<arrow::internal::Empty, void>(arrow::Status) (libarrow_python.so) #22 59.91 (C) File "???", line 0, in arrow::ConcreteFutureImpl::DoMarkFinishedOrFailed(arrow::FutureState) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void (arrow::FutureImpl const&)>::FnImpl<arrow::Future<arrow::internal::Empty>::WrapStatusyOnComplete::Callback<arrow::compute::(anonymous namespace)::ExecPlanImpl::StartProducing()::{lambda(arrow::Status const&)#3}> >::invoke(arrow::FutureImpl const&) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in void arrow::Future<arrow::internal::Empty>::MarkFinished<arrow::internal::Empty, void>(arrow::Status) (libarrow_python.so) #22 59.91 (C) File "???", line 0, in arrow::ConcreteFutureImpl::DoMarkFinishedOrFailed(arrow::FutureState) (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::internal::FnOnce<void (arrow::FutureImpl const&)>::FnImpl<arrow::Future<arrow::internal::Empty>::WrapResultyOnComplete::Callback<arrow::Future<arrow::internal::Empty>::ThenOnComplete<arrow::compute::DeclarationToStatusAsync(arrow::compute::Declaration, arrow::compute::ExecContext)::{lambda()#1}, arrow::Future<arrow::internal::Empty>::PassthruOnFailure<{lambda()#1}> > > >::~FnImpl() (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in std::_Sp_counted_ptr<arrow::compute::(anonymous namespace)::ExecPlanImpl*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::compute::(anonymous namespace)::ExecPlanImpl::~ExecPlanImpl() (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in arrow::compute::(anonymous namespace)::SourceNode::~SourceNode() (libarrow.so.1100) #22 59.91 (C) File "???", line 0, in std::_Function_handler<arrow::Future<std::optional<arrow::compute::ExecBatch> > (), arrow::MappingGenerator<arrow::dataset::EnumeratedRecordBatch, std::optional<arrow::compute::ExecBatch> > >::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (libarrow_dataset.so.1100) #22 59.91 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.91 (C) File "???", line 0, in std::_Sp_counted_ptr_inplace<arrow::MappingGenerator<arrow::dataset::EnumeratedRecordBatch, std::optional<arrow::compute::ExecBatch> >::State, std::allocator<arrow::MappingGenerator<arrow::dataset::EnumeratedRecordBatch, std::optional<arrow::compute::ExecBatch> >::State>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (libarrow_dataset.so.1100) #22 59.91 (C) File "???", line 0, in std::_Function_handler<arrow::Future<arrow::dataset::EnumeratedRecordBatch> (), arrow::ReadaheadGenerator<arrow::dataset::EnumeratedRecordBatch> >::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (libarrow_dataset.so.1100) #22 59.91 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_ptr_inplace<arrow::ReadaheadGenerator<arrow::dataset::EnumeratedRecordBatch>::State, std::allocator<arrow::ReadaheadGenerator<arrow::dataset::EnumeratedRecordBatch>::State>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Function_handler<arrow::Future<arrow::dataset::EnumeratedRecordBatch> (), arrow::MergedGenerator<arrow::dataset::EnumeratedRecordBatch> >::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_ptr_inplace<arrow::MergedGenerator<arrow::dataset::EnumeratedRecordBatch>::State, std::allocator<arrow::MergedGenerator<arrow::dataset::EnumeratedRecordBatch>::State>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Function_handler<arrow::Future<arrow::dataset::EnumeratedRecordBatch> (), arrow::MappingGenerator<arrow::Enumerated<std::shared_ptr<arrow::RecordBatch> >, arrow::dataset::EnumeratedRecordBatch> >::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_ptr_inplace<arrow::MappingGenerator<arrow::Enumerated<std::shared_ptr<arrow::RecordBatch> >, arrow::dataset::EnumeratedRecordBatch>::State, std::allocator<arrow::MappingGenerator<arrow::Enumerated<std::shared_ptr<arrow::RecordBatch> >, arrow::dataset::EnumeratedRecordBatch>::State>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Function_handler<arrow::Future<arrow::Enumerated<std::shared_ptr<arrow::RecordBatch> > > (), arrow::FutureFirstGenerator<arrow::Enumerated<std::shared_ptr<arrow::RecordBatch> > > >::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.92 (C) File "???", line 0, in arrow::ConcreteFutureImpl::~ConcreteFutureImpl() (libarrow.so.1100) #22 59.92 (C) File "???", line 0, in arrow::Future<std::function<arrow::Future<arrow::Enumerated<std::shared_ptr<arrow::RecordBatch> > > ()> >::SetResult(arrow::Result<std::function<arrow::Future<arrow::Enumerated<std::shared_ptr<arrow::RecordBatch> > > ()> >)::{lambda(void*)#1}::_FUN(void*) (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Function_handler<arrow::Future<arrow::Enumerated<std::shared_ptr<arrow::RecordBatch> > > (), arrow::EnumeratingGenerator<std::shared_ptr<arrow::RecordBatch> > >::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_ptr_inplace<arrow::EnumeratingGenerator<std::shared_ptr<arrow::RecordBatch> >::State, std::allocator<arrow::EnumeratingGenerator<std::shared_ptr<arrow::RecordBatch> >::State>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Function_handler<arrow::Future<std::shared_ptr<arrow::RecordBatch> > (), arrow::DefaultIfEmptyGenerator<std::shared_ptr<arrow::RecordBatch> > >::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_ptr_inplace<arrow::DefaultIfEmptyGenerator<std::shared_ptr<arrow::RecordBatch> >::State, std::allocator<arrow::DefaultIfEmptyGenerator<std::shared_ptr<arrow::RecordBatch> >::State>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Function_handler<arrow::Future<std::shared_ptr<arrow::RecordBatch> > (), arrow::dataset::InMemoryFragment::ScanBatchesAsync(std::shared_ptr<arrow::dataset::ScanOptions> const&)::Generator>::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (lib.cpython-311-x86_64-linux-gnu.so) #22 59.92 (C) File "???", line 0, in arrow::dataset::InMemoryFragment::~InMemoryFragment() (libarrow_dataset.so.1100) #22 59.92 (C) File "???", line 0, in arrow::SimpleRecordBatch::~SimpleRecordBatch() (libarrow.so.1100) #22 59.92 (C) File "???", line 0, in std::_Sp_counted_ptr_inplace<arrow::ArrayData, std::allocator<arrow::ArrayData>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (libarrow_python.so) #22 59.92 (C) File "???", line 0, in arrow::py::NumPyBuffer::~NumPyBuffer() (libarrow_python.so) #22 59.92 (C) File "???", line 0, in PyGILState_Ensure (libpython3.11.so.1.0) #22 59.92 (C) File "???", line 0, in pthread_exit (libc.so.6) #22 59.92 (C) File "???", line 0, in __pthread_unwind (libpthread.so.0) #22 59.92 (C) File "???", line 0, in _Unwind_ForcedUnwind (libgcc_s.so.1) #22 59.92 (C) File "???", line 0, in __gxx_personality_v0 (libstdc++.so.6) #22 59.92 (C) File "???", line 0, in abort (libc.so.6) #22 59.92 (C) File "???", line 0, in raise (libc.so.6) #22 59.92 #22 59.92 Traceback for thread 21 [] (most recent call last): #22 59.92 (C) File "???", line 0, in __clone (libc.so.6) #22 59.92 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.92 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.92 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.92 #22 59.92 Traceback for thread 31 [] (most recent call last): #22 59.92 (C) File "???", line 0, in __clone (libc.so.6) #22 59.92 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.92 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.92 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.92 #22 59.92 Traceback for thread 20 [] (most recent call last): #22 59.92 (C) File "???", line 0, in __clone (libc.so.6) #22 59.92 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.92 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.92 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.92 #22 59.92 Traceback for thread 18 [] (most recent call last): #22 59.92 (C) File "???", line 0, in __clone (libc.so.6) #22 59.92 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.92 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.92 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.92 #22 59.92 Traceback for thread 19 [] (most recent call last): #22 59.92 (C) File "???", line 0, in __clone (libc.so.6) #22 59.92 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.92 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 22 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 23 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 26 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 24 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 25 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 27 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 32 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 28 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 38 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.93 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.93 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 29 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.93 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.93 #22 59.93 Traceback for thread 40 [] (most recent call last): #22 59.93 (C) File "???", line 0, in __clone (libc.so.6) #22 59.93 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.93 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.93 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.94 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.94 #22 59.94 Traceback for thread 53 [] (most recent call last): #22 59.94 (C) File "???", line 0, in __clone (libc.so.6) #22 59.94 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.94 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.94 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.94 #22 59.94 Traceback for thread 57 [] (most recent call last): #22 59.94 (C) File "???", line 0, in __clone (libc.so.6) #22 59.94 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.94 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.94 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.94 #22 59.94 Traceback for thread 49 [] (most recent call last): #22 59.94 (C) File "???", line 0, in __clone (libc.so.6) #22 59.94 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.94 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.94 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.94 #22 59.94 Traceback for thread 39 [] (most recent call last): #22 59.94 (C) File "???", line 0, in __clone (libc.so.6) #22 59.94 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.94 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.94 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.94 #22 59.94 Traceback for thread 50 [] (most recent call last): #22 59.94 (C) File "???", line 0, in __clone (libc.so.6) #22 59.94 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.94 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.94 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.94 #22 59.94 Traceback for thread 30 [] (most recent call last): #22 59.94 (C) File "???", line 0, in __clone (libc.so.6) #22 59.94 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.94 (C) File "???", line 0, in blas_thread_server (libopenblas64_p-r0-15028c96.3.21.so) #22 59.94 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.94 #22 59.94 Traceback for thread 47 [] (most recent call last): #22 59.94 (C) File "???", line 0, in __clone (libc.so.6) #22 59.94 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.94 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.94 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.94 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.94 #22 59.94 Traceback for thread 43 [] (most recent call last): #22 59.94 (C) File "???", line 0, in __clone (libc.so.6) #22 59.94 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.94 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.95 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.95 #22 59.95 Traceback for thread 54 [] (most recent call last): #22 59.95 (C) File "???", line 0, in __clone (libc.so.6) #22 59.95 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.95 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.95 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.95 #22 59.95 Traceback for thread 56 [] (most recent call last): #22 59.95 (C) File "???", line 0, in __clone (libc.so.6) #22 59.95 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.95 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.95 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.95 #22 59.95 Traceback for thread 48 [] (most recent call last): #22 59.95 (C) File "???", line 0, in __clone (libc.so.6) #22 59.95 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.95 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.95 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.95 #22 59.95 Traceback for thread 44 [] (most recent call last): #22 59.95 (C) File "???", line 0, in __clone (libc.so.6) #22 59.95 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.95 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.95 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.95 #22 59.95 Traceback for thread 51 [] (most recent call last): #22 59.95 (C) File "???", line 0, in __clone (libc.so.6) #22 59.95 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.95 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.95 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.95 #22 59.95 Traceback for thread 41 [] (most recent call last): #22 59.95 (C) File "???", line 0, in __clone (libc.so.6) #22 59.95 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.95 (C) File "src/background_thread.c", line 444, in background_thread_entry (libarrow.so.1100) #22 59.95 (C) File "src/background_thread.c", line 419, in background_work (inlined) (libarrow.so.1100) #22 59.95 (C) File "src/background_thread.c", line 229, in background_work_sleep_once (inlined) (libarrow.so.1100) #22 59.95 (C) File "src/background_thread.c", line 159, in background_thread_sleep (inlined) (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.95 #22 59.95 Traceback for thread 45 [] (most recent call last): #22 59.95 (C) File "???", line 0, in __clone (libc.so.6) #22 59.95 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.95 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.95 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.95 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.95 #22 59.95 Traceback for thread 42 [] (most recent call last): #22 59.96 (C) File "???", line 0, in __clone (libc.so.6) #22 59.96 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.96 (C) File "src/background_thread.c", line 444, in background_thread_entry (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 419, in background_work (inlined) (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 229, in background_work_sleep_once (inlined) (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 159, in background_thread_sleep (inlined) (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.96 #22 59.96 Traceback for thread 55 [] (most recent call last): #22 59.96 (C) File "???", line 0, in __clone (libc.so.6) #22 59.96 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.96 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.96 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.96 #22 59.96 Traceback for thread 33 [] (most recent call last): #22 59.96 (C) File "???", line 0, in __clone (libc.so.6) #22 59.96 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.96 (C) File "src/background_thread.c", line 444, in background_thread_entry (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 412, in background_work (inlined) (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 374, in background_thread0_work (inlined) (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 229, in background_work_sleep_once (inlined) (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 159, in background_thread_sleep (inlined) (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.96 #22 59.96 Traceback for thread 58 [] (most recent call last): #22 59.96 (C) File "???", line 0, in __clone (libc.so.6) #22 59.96 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.96 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.96 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.96 #22 59.96 Traceback for thread 52 [] (most recent call last): #22 59.96 (C) File "???", line 0, in __clone (libc.so.6) #22 59.96 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.96 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.96 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.96 #22 59.96 Traceback for thread 46 [] (most recent call last): #22 59.96 (C) File "???", line 0, in __clone (libc.so.6) #22 59.96 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.96 (C) File "???", line 0, in execute_native_thread_routine (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in std::condition_variable::wait(std::unique_lock<std::mutex>&) (libstdc++.so.6) #22 59.96 (C) File "???", line 0, in pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.96 #22 59.96 Traceback for thread 37 [] (most recent call last): #22 59.96 (C) File "???", line 0, in __clone (libc.so.6) #22 59.96 (C) File "???", line 0, in start_thread (libpthread.so.0) #22 59.96 (C) File "src/background_thread.c", line 444, in background_thread_entry (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 419, in background_work (inlined) (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 229, in background_work_sleep_once (inlined) (libarrow.so.1100) #22 59.96 (C) File "src/background_thread.c", line 159, in background_thread_sleep (inlined) (libarrow.so.1100) #22 59.96 (C) File "???", line 0, in pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0) #22 59.96 #22 DONE 60.1s ``` Another one: ```console #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007f7ac74f5537 in __GI_abort () at abort.c:79 #2 0x00007f7a97cb07ec in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #3 0x00007f7a97cbb966 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #4 0x00007f7a97cbb9d1 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #5 0x00007f7a97cbb3cc in __gxx_personality_v0 () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #6 0x00007f7ac42478a4 in ?? () from /lib/x86_64-linux-gnu/libgcc_s.so.1 #7 0x00007f7ac4247f4e in _Unwind_ForcedUnwind () from /lib/x86_64-linux-gnu/libgcc_s.so.1 #8 0x00007f7ac74c2c30 in __GI___pthread_unwind (buf=<optimized out>) at unwind.c:121 #9 0x00007f7ac74ba18c in __do_cancel () at pthreadP.h:310 #10 __pthread_exit (value=<optimized out>) at pthread_exit.c:28 #11 0x00007f7ac75537f4 in __pthread_exit (retval=<optimized out>) at forward.c:141 #12 0x00007f7ac7813ab8 in ?? () from /usr/local/bin/../lib/libpython3.11.so.1.0 #13 0x00007f7ac7b69d00 in _PyRuntime () from /usr/local/bin/../lib/libpython3.11.so.1.0 #14 0x0000000000001388 in ?? () #15 0x0000000000000000 in ?? () ``` </details> # Workaround I've attempted to lower the `ARROW_IO_THREADS` and `OMP_NUM_THREADS` but it doesn't have an effect. Setting `use_threads=False` takes care of the problem but is quite non-performant. ### Component(s) C++, Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
