zuston commented on issue #378: URL: https://github.com/apache/incubator-uniffle/issues/378#issuecomment-1344260350
> > I feel that it's necessary. Because it will be too slow that the server only use single thread to flush a large partition although we have multiple servers. > > Make sense. I think I will do this optimization. Draft a PR #396 . For this PR, it only solve the problem mentioned by second case. But if the `ShuffleDataFlushEvent` is so huge(5G) to HDFS, it's still handled by single one thread. So we also need to generate multiple events from huge `ShuffleDataFlushEvent` to improve writing speed when flushing to HDFS. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@uniffle.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
