zuston commented on issue #674: URL: https://github.com/apache/incubator-uniffle/issues/674#issuecomment-1454611218
> This sounds good to me. Do you apply this in your prod env or not? No. I have to say after applying this optimization, the GC pause is still serious in my point to point writing benchmark. So maybe this is not the main problem. Anyway, this is still an improvement. BTW, I want to share the benchmark result and I think uniffle's users will benefit from this. Shuffle server environment 1. XMX 20G 2. buffer capacity 10G 3. read capacity 5G Shuffle Writer program 1. using spark to write shuffle data with 20 executors. Single executor will total write 1G, and each time write 14M | Java version | ShuffleServer GC | Max pause time | ThroughOutput | | ------------- | ------------- | ------------- | ------------- | | 8 | G1 | 30s | 0.3 | | 11 | G1 | 2.5s | 0.8 | | 18 | G1 | 2.5s | 0.8 | | 18 | ZGC | 0.2ms | 0.99997 | It's amazing for me. I think we should drop java8 version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@uniffle.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
