skyyyy955 commented on issue #10458:
URL: https://github.com/apache/paimon/issues/10458#issuecomment-7056678684

   I ran `CompactCandidateRangeCollector` at `7a30ef580e` with the 12 × 200 MiB 
metadata layout from the report (512 MiB target, 4 MiB open cost). It selected 
no ranges with `compaction.min.file-num=5`. Changing only that setting to `2` 
produced four ranges of three files each. The [collector 
source](https://github.com/apache/paimon/blob/ee544b74f172fcb14274812161e6b411a66a2046/paimon-core/src/main/java/org/apache/paimon/append/dataevolution/CompactCandidateRangeCollector.java#L541-L550)
 is identical on current master (`ee544b74f1`).
   
   I also wrote 12 batches to a local Data Evolution table and read back all 
12,000 row IDs. Its `$files` metadata showed 12 Parquet files of 210,014 bytes 
each with contiguous row IDs. Feeding those values to the collector with a 512 
KiB target and a 4 KiB open cost again gave zero ranges at a minimum of five 
files and four ranges at a minimum of two. Both checks stop at candidate 
selection, before any Spark or Flink compaction job rewrites files.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to