timvw commented on code in PR #2394:
URL: https://github.com/apache/arrow-datafusion/pull/2394#discussion_r865263570
##########
data-access/src/object_store/mod.rs:
##########
@@ -102,3 +132,118 @@ pub trait ObjectStore: Sync + Send + Debug {
/// Get object reader for one file
fn file_reader(&self, file: SizedFile) -> Result<Arc<dyn ObjectReader>>;
}
+
+const GLOB_START_CHARS: [char; 3] = ['?', '*', '['];
+
+/// Determine whether the path contains a globbing character
+fn contains_glob_start_char(path: &str) -> bool {
+ path.chars().any(|c| GLOB_START_CHARS.contains(&c))
+}
+
+/// Filters the file_stream to only contain files that end with suffix
+fn filter_suffix(file_stream: FileMetaStream, suffix: &str) ->
Result<FileMetaStream> {
+ let suffix = suffix.to_owned();
+ Ok(Box::pin(file_stream.filter(move |fr| {
Review Comment:
This is the code that was already in place.
When I update the code to use try_filter, it will not pass-through the
errors anymore (not sure if anyone was actually doing something with them
though..)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
