jerqi commented on issue #378: URL: https://github.com/apache/incubator-uniffle/issues/378#issuecomment-1343770114
> > I mean that we can write multiple files at the same time. > > We can tryLock the file lock, we have multiple locks, if we succeed to tryLock, we can write the file. If we fail to tryLock, we will retry another file lock. > > Got it. It's an optional solution, the flush speed could improve based on (HDFS speed) * (file number). > > And I think we also need to limit the huge partition app's writing speed to avoid affecting other jobs. VIP Shop have similarly minds. cc @Gustfh -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@uniffle.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
