JingsongLi commented on code in PR #1465:
URL: https://github.com/apache/paimon-rust/pull/1465#discussion_r4643305086


##########
crates/paimon/src/table/data_file_writer.rs:
##########
@@ -317,7 +317,6 @@ impl DataFileWriter {
             .new_file_name(&self.data_file_prefix, &self.file_format);
         let location = self.paths.new_path(&file_name)?;
         let bucket_dir = location.parent();
-        self.file_io.mkdirs(&format!("{bucket_dir}/")).await?;

Review Comment:
   **[P2] Keep parent creation for providers that require it**
   
   This removes the directory creation step for every `FileIO`, including 
existing custom `FileIOProvider`s. I used a public `TableWrite` with a provider 
backed by a hierarchical OpenDAL filesystem that requires the parent directory 
before resolving a Parquet file. Writing the first row of a fresh partition and 
bucket prepared a real Parquet file on `589e7743`. On `2ea74d4d` it fails 
because `pt=a%2Fb/bucket-0` does not exist. The KV writer drops the same 
prerequisite.
   
   Keep the current behavior for provider-backed or unclassified storage. Skip 
the marker only for backends known to create parents on write or accept object 
keys without directories.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to