wjones127 commented on code in PR #2246:
URL: https://github.com/apache/arrow-datafusion/pull/2246#discussion_r859314973
##########
ballista/rust/core/src/serde/logical_plan/mod.rs:
##########
@@ -1016,6 +1016,55 @@ mod roundtrip_tests {
"this is only a test object store".to_string(),
))
}
+
+ fn file_writer(
+ &self,
+ _path: &str,
+ ) -> datafusion_data_access::Result<Arc<dyn ObjectWriter>> {
+ unimplemented!();
+ }
+
+ async fn create_dir(
Review Comment:
It looks like fsspec implementation [doesn't create empty objects as
directories](https://github.com/fsspec/s3fs/blob/c4fb41f7cc2f2aede6bbb7755096c38b9e4cc553/s3fs/core.py#L730-L761),
so I think it is sensible to do the same.
It's not totally a no-op though for cases like S3. You might want to have
`create_dir` create a bucket if it doesn't exist (though I'd like for there to
be an option to turn off bucket creation). And it should probably check that
there isn't an existing file at the path you are trying to create a directory
at.
I think what I will do is document the expected behavior and create a
generic test that can be run on an arbitrary filesystem to help implementation
make sure they follow expected behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
