awdavidson opened a new pull request, #914: URL: https://github.com/apache/incubator-uniffle/pull/914
What changes were proposed in this pull request? Ensure all events are marked as understorage, this will result to the LocalStorageMeta being updated when events are processed. Why are the changes needed? Currently LocalStorageMeta is only update with metrics from the first event in a given shuffleId and partitionId, the first event updates metrics because there is no entry in partitionsOfStorage and the event get marked as underStorage, however, for future events in the same shuffleId and partitionId selectStorage returns the storage and does not mark the event as underStorage so when updateWriteMetrics is called, event.getUnderStorage() returns null and storage.updateWriteMetrics(metrics); is skipped. As metrics are not updated correctly, LocalStorage.canWrite will not return the correct result. Does this PR introduce any user-facing change? No. How was this patch tested? Added unit test which covers multi events for the same shuffleId and partitionId -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
