sp-202 opened a new pull request, #22900: URL: https://github.com/apache/datafusion/pull/22900
This PR enhances observability of CSV file reading operations by adding comprehensive metrics collection. These metrics enable better performance monitoring and debugging of CSV datasource operations during query execution, helping identify bottlenecks and optimize I/O operations. ## What changes are included in this PR? - Implement `as_any()` method on `CsvSource` for dynamic casting support - Add bytes scanned counter to track the amount of data fetched from the object store for each partition - Record output metrics (row count) for file-based CSV reading path (non-wasm32) - Record output metrics (row count) for stream-based CSV reading path - These metrics integrate with the existing baseline metrics infrastructure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
