advancedxy commented on PR #428: URL: https://github.com/apache/incubator-uniffle/pull/428#issuecomment-1362680429
> > Yeah, that's possible.... But the flush trigger is not non-deterministic anyway, it's trigged by outside's requests, which may vary. I still don't get why this kind of change would affect `LOCAL_ORDER` optimization or bring performance penalty here. > > 1.If `preAllocatedSize` release is slow, the client write speed will be affected. When I did #159 , i encountered similar problems. 2.`LOCAL_ORDER` will affect performance when cache data, but i am not sure how much impact. In general, there will be a lot of performance impact if this place is not handled well. So, i think simple test is needed. 1. From my understanding of the code of shuffleBufferManager, `preAllocatedSize` won't back-pressure shuffle write client, `usedMemory` does. However `usedMemory` doesn't decrease when `preAllocatedSize` decreased. 2. `preAllocatedSize` may affect the flush behavior, as it's decreased slightly slower. As mentioned in previous reply, it would cause flush less frequently. `LocalOrder` would be affected when caching data but flush is more frequently triggered. 3. If it in theory may cause some performance issues, a simple test may be sufficient as it requires complexes server environments to simulate. In short, I'm not convinced that this change would make a big difference here and affects local_order. cc @zuston or @jerqi do you any other input on this? But the behavior is indeed slightly different, I may make an amend to sync back to the original behavior and makes every one happy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
