jadewang-db opened a new pull request, #2747:
URL: https://github.com/apache/arrow-adbc/pull/2747
### Summary
This PR fixes an issue where PowerBI would hang when reading CloudFetch
results and significantly improves the logging capabilities in the CloudFetch
downloader component.
### Problem
1. The CloudFetchReader was not properly disposing of the download manager
after completing downloads, causing resource leaks that led to PowerBI hanging.
2. The CloudFetchDownloader was using Debug.WriteLine for logging, which is
inadequate for production scenarios and doesn't provide sufficient diagnostic
information.
### Solution
- Fixed resource management in CloudFetchReader by properly disposing the
download manager after all files are processed
- Replaced Debug.WriteLine calls with more comprehensive Trace logging
- Added detailed performance metrics and diagnostics:
- Download start/completion timestamps
- File sizes and throughput calculations
- Decompression metrics
- Overall download statistics (total files, success/failure counts)
- Added URL sanitization for secure logging
- Added proper error tracking and reporting
### Testing
- Enhanced CloudFetchE2ETest to verify that the reader properly completes
after all data is consumed
- Verified that PowerBI no longer hangs when reading CloudFetch results
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
