alamb commented on code in PR #8565:
URL: https://github.com/apache/arrow-datafusion/pull/8565#discussion_r1431835143
##########
datafusion/core/src/datasource/listing/url.rs:
##########
@@ -424,6 +433,13 @@ mod tests {
let b = ListingTableUrl::parse("../bar/./foo/../baz").unwrap();
assert_eq!(a, b);
assert!(a.prefix.as_ref().ends_with("bar/baz"));
+
+ let url = ListingTableUrl::parse("../foo/*.parquet").unwrap();
Review Comment:
I am probably missing something here, but how does this test the new code? I
don't see it passing in `ignore_subdirectory`
##########
datafusion/core/src/execution/context/parquet.rs:
##########
@@ -109,7 +109,7 @@ mod tests {
.read_parquet(
// it was reported that when a path contains // (two
consecutive separator) no files were found
// in this test, regardless of parquet_test_data() value, our
path now contains a //
- format!("{}/..//*/alltypes_plain*.parquet",
parquet_test_data()),
Review Comment:
I don't fully understand this question
##########
datafusion/sqllogictest/test_files/information_schema.slt:
##########
@@ -150,6 +150,7 @@ datafusion.execution.aggregate.scalar_update_factor 10
datafusion.execution.batch_size 8192
datafusion.execution.coalesce_batches true
datafusion.execution.collect_statistics false
+datafusion.execution.ignore_subdirectory true
Review Comment:
Could we use a name that gives some context about when the
`ignore_subdirectory` is actually used?
For example, maybe like `listing_table_ignore_subdirectory`. Or maybe even
it is time to create a whole new category of configuration for listing tables
`datafusion.listing_table.ignore_subdirectory` 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
