JingsongLi commented on code in PR #1329:
URL: https://github.com/apache/paimon-rust/pull/1329#discussion_r4478750976


##########
docs/src/c-binding.md:
##########
@@ -802,6 +802,12 @@ batches, file indexes, caller-owned Arrow input, and 
allocation overhead are
 not fully charged. A reservation failure does not automatically flush or retry
 a writer; handle the error as a failed write operation.
 
+For formats that support flushing, a data file also flushes when its own
+retained-input reservation reaches `write-buffer-size`, even if encoding (for

Review Comment:
   [P2] Document the buffer option that controls this flush
   
   The public table writer gets the threshold from 
`CoreOptions::write_parquet_buffer_size()`, which reads 
`write.parquet-buffer-size` and defaults to 256 MiB. The documented 
`write-buffer-size` key is read by the sorted global-index builder and leaves 
this file-writer threshold at its default.
   
   I reproduced the difference through 
`Table::new_write_builder().with_resources(...)` with a 6 MiB budget and 
distinct 4 MiB dictionary-friendly batches. With the documented key set to 1 
MiB, the second write fails with `ResourceExhausted`. With 
`write.parquet-buffer-size=1 MB`, all four batches write, commit, and scan back 
as 4,096 rows with the expected payload.
   
   Please use `write.parquet-buffer-size` here and cover the option through the 
public table writer so the test also exercises configuration wiring.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to