LuciferYang opened a new pull request, #10938:
URL: https://github.com/apache/paimon/pull/10938

   ### Purpose
   
   close #10937
   
   Data evolution compaction (`CALL sys.compact` and `CALL 
sys.materialize_deletion_vectors`) creates its append writer without an 
`IOManager`. With `write-buffer-for-append = true`, the first spill of the 
write buffer failed with a `NullPointerException` in `ExternalBuffer.spill`.
   
   `DataEvolutionCompactTask` now always sets `write-buffer-for-append = false` 
for its writers, which the split-large-files branch of the normal task already 
did. The buffer only holds rows before they go to the same rolling writer, so 
writing them directly gives the same output.
   
   ### Tests
   
   Two tests in `DataEvolutionDeletionTestBase`, "compact with spillable append 
write buffer" and "materialize deletion vectors with spillable append write 
buffer". Each table has `write-buffer-for-append` with a 256 kb buffer and two 
files of about 1 MB, so the compaction writer has to spill. They check the 
rows, the row ids and, for compact, that one file is left. On master both fail 
with the `NullPointerException`.
   
   ### API and Format
   
   no
   
   ### Documentation
   
   no
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to