houqp commented on a change in pull request #932:
URL: https://github.com/apache/arrow-datafusion/pull/932#discussion_r698038625
##########
File path: datafusion/src/datasource/mod.rs
##########
@@ -36,3 +43,247 @@ pub(crate) enum Source<R = Box<dyn std::io::Read + Send +
Sync + 'static>> {
/// Read data from a reader
Reader(std::sync::Mutex<Option<R>>),
}
+
+#[derive(Debug, Clone)]
+/// A single file that should be read, along with its schema, statistics
+/// and partition column values that need to be appended to each row.
+pub struct PartitionedFile {
+ /// Path for the file (e.g. URL, filesystem path, etc)
+ pub file_path: String,
Review comment:
I think the convention is to use uri for strings that could contain a
uri scheme components like `s3://`, `gcs://`, etc. For object store, it uses
path because uri scheme is already implicitly assumed by the object store
implementation, so it's not needed anymore. On top of that, it avoids the need
to perform uri validation in object store, for example, rejecting uris with
`gcs://` schemes in s3 object store implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
