JingsongLi commented on code in PR #1465:
URL: https://github.com/apache/paimon-rust/pull/1465#discussion_r4643305086
##########
crates/paimon/src/table/data_file_writer.rs:
##########
@@ -317,7 +317,6 @@ impl DataFileWriter {
.new_file_name(&self.data_file_prefix, &self.file_format);
let location = self.paths.new_path(&file_name)?;
let bucket_dir = location.parent();
- self.file_io.mkdirs(&format!("{bucket_dir}/")).await?;
Review Comment:
**[P2] Keep parent creation for providers that require it**
This removes the directory creation step for every `FileIO`, including
existing custom `FileIOProvider`s. I used a public `TableWrite` with a provider
backed by a hierarchical OpenDAL filesystem that requires the parent directory
before resolving a Parquet file. Writing the first row of a fresh partition and
bucket prepared a real Parquet file on `589e7743`. On `2ea74d4d` it fails
because `pt=a%2Fb/bucket-0` does not exist. The KV writer drops the same
prerequisite.
Keep the current behavior for provider-backed or unclassified storage. Skip
the marker only for backends known to create parents on write or accept object
keys without directories.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]