comphead commented on code in PR #20592:
URL: https://github.com/apache/datafusion/pull/20592#discussion_r2875380049
##########
datafusion/datasource/src/file_stream.rs:
##########
@@ -399,6 +413,14 @@ pub struct FileStreamMetrics {
/// If using `OnError::Skip` this will provide a count of the number of
files
/// which were skipped and will not be included in the scan results.
pub file_scan_errors: Count,
+ /// Count of files successfully opened.
+ pub files_opened: Count,
+ /// Count of files closed (opened, pruned, or skipped due to limit).
+ /// When the stream completes, this equals the total number of files
+ /// assigned to this partition.
+ pub files_closed: Count,
+ /// Count of files completely scanned (reader stream fully consumed).
+ pub files_scanned: Count,
Review Comment:
would be that more explanatory to see how many files were skipped instead of
closed? Intuitively i would suggest number of open files == number of closed
files, unless we debug a connection leak. And skipped by stats would make more
sense?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
