Neuw84 commented on issue #17209: URL: https://github.com/apache/iceberg/issues/17209#issuecomment-4982445862
Hi @RussellSpitzer, As I was doing benchmark on V2 on V3 I forced an scenario where all the partitions are touched. Therefore, it was compaction that was messing for the tables ( that´s why I closed the issue). However, I have seen random spikes, executors dying and tasks with high deserialization time on V3 afterwards... I have spent the whole morning trying to see what is happening but without any luck. This is an example of what I see ( the first huge spike is just compaction on S3 Tables). <img width="2638" height="1246" alt="Image" src="https://github.com/user-attachments/assets/764bae0f-104a-49e6-b9eb-8f627d2d9d93"; /> The same workload with V2 is completely stable ( just the performance is slower with time because positional deletes accumulation). V3 numbers are a lot better, but with those spikes is not usable. Will try to reopen with more information or a more concrete reproducible example on OSS Spark. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
