lidavidm edited a comment on pull request #11964:
URL: https://github.com/apache/arrow/pull/11964#issuecomment-995043666
TODO:
- [ ] Confirm that we can link these spans to another parent span
I originally added a parameter to ScanOptions but this won't work since it
would expose OpenTelemetry symbols in the public API.
- [ ] AddressSanitizer reports some leaks, either we need to add a
suppression and report upstream or work out why we're getting failures.
I've added suppressions in case they help others, but I can't quite work
out why it happens in the first place. Manually shutting down the thread pool
results in no leaks. With the debugger I can confirm the thread pool is
shutting itself down at thread exit, but for some reason, in the second case,
thread local destructors don't seem to get run. It resembles this [LLVM
thread](https://lists.llvm.org/pipermail/cfe-users/2015-December/000836.html)
(though there's no follow up there) - this might be something we should
minimize and submit upstream.
Weirdly, adding a span on the main thread fixes this...
Example of output:
<details>
```
$ env ARROW_TRACING_BACKEND=ostream ./debug/arrow-dataset-file-ipc-test
--gtest_filter='TestScan/*.ScanRecordBatchReader/0AsyncThreaded16b1024r'
Running main() from
/home/lidavidm/Code/upstream/arrow-15067/build/googletest_ep-prefix/src/googletest_ep/googletest/src/gtest_main.cc
Note: Google Test filter =
TestScan/*.ScanRecordBatchReader/0AsyncThreaded16b1024r
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from TestScan/TestIpcFileFormatScan
[ RUN ]
TestScan/TestIpcFileFormatScan.ScanRecordBatchReader/0AsyncThreaded16b1024r
[ OK ]
TestScan/TestIpcFileFormatScan.ScanRecordBatchReader/0AsyncThreaded16b1024r
(418 ms)
[----------] 1 test from TestScan/TestIpcFileFormatScan (418 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (418 ms total)
[ PASSED ] 1 test.
{
name : arrow::dataset::IpcFileFormat::OpenReaderAsync
trace_id : 483084a5464d0ef6278da1cc982f9dd9
span_id : beb598d1dcb93984
tracestate :
parent_span_id: b8ec67d5f5fbc11d
start : 1639591037854068113
duration : 2869368
description :
span kind : Internal
status : Ok
attributes :
thread_id: 139972187428928
events :
links :
resources :
service.name: unknown_service
telemetry.sdk.version: 1.1.0
telemetry.sdk.name: opentelemetry
telemetry.sdk.language: cpp
instr-lib : arrow
}
{
name : arrow::dataset::IpcFileFormat::ScanBatchesAsync::Next
trace_id : 483084a5464d0ef6278da1cc982f9dd9
span_id : 6462f59e9d0532df
tracestate :
parent_span_id: b8ec67d5f5fbc11d
start : 1639591037864180748
duration : 3811981
description :
span kind : Internal
status : Ok
attributes :
thread_id: 139972187428928
events :
links :
resources :
service.name: unknown_service
telemetry.sdk.version: 1.1.0
telemetry.sdk.name: opentelemetry
telemetry.sdk.language: cpp
instr-lib : arrow
}
(snip)
{
name : arrow::dataset::FragmentToBatches
trace_id : 483084a5464d0ef6278da1cc982f9dd9
span_id : b8ec67d5f5fbc11d
tracestate :
parent_span_id: 0000000000000000
start : 1639591037853521954
duration : 169164805
description :
span kind : Internal
status : Ok
attributes :
thread_id: 139972070205184
arrow.dataset.fragment.type_name: ipc
arrow.dataset.fragment.last: 1
arrow.dataset.fragment.index: 0
arrow.dataset.fragment: <Buffer>
events :
links :
resources :
service.name: unknown_service
telemetry.sdk.version: 1.1.0
telemetry.sdk.name: opentelemetry
telemetry.sdk.language: cpp
instr-lib : arrow
}
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
