shyjsarah commented on code in PR #707:
URL: https://github.com/apache/paimon-rust/pull/707#discussion_r3773971831
##########
crates/paimon/src/table/vindex_index_build_builder.rs:
##########
@@ -245,13 +431,38 @@ impl<'a> VindexIndexBuildBuilder<'a> {
self.table.location().trim_end_matches('/'),
file_name
);
- self.table
- .file_io()
- .new_output(&index_path)?
- .write(Bytes::from(bytes))
- .await?;
-
- let status = self.table.file_io().get_status(&index_path).await?;
+ let write_result = async {
+ let async_writer = self
+ .table
+ .file_io()
+ .new_output(&index_path)?
+ .async_writer()
+ .await?;
+ let mut output = SyncIoBridge::new(async_writer);
+ tokio::task::spawn_blocking(move || -> std::io::Result<()> {
+ let mut writer = writer;
+ writer.write(&mut PosWriter::new(&mut output))?;
+ output.shutdown()
Review Comment:
**Suggestion:** Since a running `spawn_blocking` task cannot be aborted,
cancelling the outer build future may still allow this worker to finish
`output.shutdown()` after the surrounding cleanup path has been dropped. The
resulting index file would not be referenced by any commit message or manifest.
If orphan index files are already covered by a separate cleanup mechanism,
could we document or test that assumption here? Otherwise, it may be worth
keeping a provisional output guard or another cleanup mechanism until the file
is transferred into a `CommitMessage`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
