timvw opened a new issue, #3261:
URL: https://github.com/apache/arrow-datafusion/issues/3261
**Is your feature request related to a problem or challenge? Please describe
what you are trying to do.**
Since the much needed cleanup and rationalization in #2578 of
ListingTableUrl there is only support for glob patterns when no scheme is
provided (in practice: only supported on local filesystem and not on other
object_stores anymore).
**Describe the solution you'd like**
To have proper support for glob patterns. Eg, updating the documentation
(and implementation) for ListingTableUrl to the following:
/// Parse a provided string as a `ListingTableUrl`
///
/// # Glob File Paths
///
/// If the path contains any of `'?', '*', '['`, it will be considered
/// a glob expression and resolved as following:
///
/// The string up to the first path segment containing a glob expression
will be extracted,
/// and resolved as any other provided string.
///
/// The remaining string will be interpreted as a [`glob::Pattern`] and
used as a
/// filter when listing files from object storage
///
/// # Paths without a Scheme
///
/// If no scheme is provided, or the string is an absolute filesystem
path
/// as determined [`std::path::Path::is_absolute`], the string will be
/// interpreted as a path on the local filesystem using the operating
/// system's standard path delimiter, i.e. `\` on Windows, `/` on Unix.
///
/// If you wish to specify a path that does not exist on the local
/// machine you must provide it as a fully-qualified [file URI]
/// e.g. `file:///myfile.txt`
///
/// [file URI]: https://en.wikipedia.org/wiki/File_URI_scheme
**Describe alternatives you've considered**
We could keep things as they are and push support for globbing further into
user-space.
Today, when a path/string contains an '*' or '[' the user is greeted with a
BadSegment error anyway.
@tustvold WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
