QuakeWang commented on code in PR #1465:
URL: https://github.com/apache/paimon-rust/pull/1465#discussion_r4639583034


##########
crates/paimon/src/table/data_file_writer.rs:
##########
@@ -317,7 +317,6 @@ impl DataFileWriter {
             .new_file_name(&self.data_file_prefix, &self.file_format);
         let location = self.paths.new_path(&file_name)?;
         let bucket_dir = location.parent();
-        self.file_io.mkdirs(&format!("{bucket_dir}/")).await?;

Review Comment:
   Could we skip `mkdirs()` for S3 in `FileIO` and keep it for backends that 
need it? I tested a `FileIOProvider` backed by OpenDAL WebDAV with automatic 
parent creation disabled. Both append and primary-key writes to a new 
`bucket-0` failed with HTTP 409 during `prepare_commit()`. The same writes 
passed after explicitly creating the bucket directory.
   
   Could we also add a regression test that checks both S3 write paths produce 
a Parquet file without sending a directory marker PUT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to