JingsongLi commented on PR #736: URL: https://github.com/apache/paimon-rust/pull/736#issuecomment-5419881521
[P2] Restore a bounded upload path or drop the multipart claim At the current head aa9eb0e, vindex_index_build_builder.rs still calls async_writer(), which delegates to async_writer_with_concurrency(1). The new async_writer_with_concurrency helper has no caller requesting concurrency greater than 1; commit 23e352d reverted the only concurrency=4 call. As a result, vector-index uploads still serialize their 8 MiB parts, so an index larger than one part continues to pay one upload round trip per part. The PR title, Changes section, and the 30.3s to 4.6s benchmark therefore no longer describe the code being reviewed. Please either add a genuinely bounded concurrent upload implementation and use it here, or remove the unused helper and update the title, description, and benchmark scope to cover only the Parquet-read improvement. Simply restoring the previous OpenDAL concurrency setting would reintroduce the unbounded buffering issue already identified in this review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
