Hi Grant, Not yet. The benchmark so far measures the conversion alone, with no other writer on the table.
There are two cases: - If the compaction commits first, the sink's commit fails, and the coordinator converts again and retries. - If the sink commits first, a compaction that is rewriting one of those files fails. The sink cannot solve this on its own. When the changes are spread over the table, I expect a rewrite that takes longer than the commit interval to rarely succeed. I'll run compaction alongside repeated conversion commits and report the retries, the commit latency, and how often the compaction succeeds. I'm away until November 28, so I expect to share the results in the first half of December. Thanks, Koodin 2026년 11월 14일 (토) 오전 12:47, Grant Nicholas <[email protected]>님이 작성: > > - koodin9 posted a benchmark on #18003 [2]: with increasing IDs a > conversion stays under 0.4 s, while random UUID keys on a 10,000-file table > read every file and take 2 to 42 s depending on storage latency. > > Were you able to test performance under commit contention with other > writes (e.g. compaction)? > > Writing deletion vectors instead of equality deletes can generate > significant contention. For delete-heavy low-latency commit workloads like > Kafka Connect CDC, contention can result in many commit retries, increased > latency, and decreased throughput. > > For context, my coworker Pachu wrote a blog post describing the > contention in more detail > <https://www.starburst.io/blog/equality-deletes-compaction-and-iceberg-v4/> >
