GitHub user moba15 added a comment to the discussion: Arrow-Datasets C++: Scanner::Scan visitor is executing serially despite use_threads=true
Sorry for the late response: For me I found it quite confusing that the documentation states > If multiple threads are used (via use_threads), the visitor will be invoked > from those threads and is responsible for any synchronization. for Sanner::Scan. To me, the phrase "responsible for any synchronization" heavily implied that if use_threads=true, multiple visitor callbacks could execute concurrently at the same time. However, thanks to the explanations here, I now understand that the calls remain strictly sequential (single-consumer), and the warning just means the callback might be invoked from different worker threads across different batches. It might clarify things for future users if the docs explicitly stated that the visitor is still executed serially. For example, it could be updated to something like: "If multiple threads are used (via use_threads), the visitor may be invoked from different worker threads. However, the visitor is guaranteed to be called sequentially (one batch at a time). The user is responsible for thread-safety if the visitor shares state across these different thread contexts." GitHub link: https://github.com/apache/arrow/discussions/49568#discussioncomment-17825043 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
