awdavidson opened a new pull request, #902:
URL: https://github.com/apache/incubator-uniffle/pull/902
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
Examples:
- "[#123] feat(operator): support xxx"
- "[#233] fix: check null before access result in xxx"
- "[MINOR] refactor: fix typo in variable name"
- "[MINOR] docs: fix typo in README"
- "[#255] test: fix flaky test NameOfTheTest"
Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. Contributor guidelines:
https://github.com/apache/incubator-uniffle/blob/master/CONTRIBUTING.md
3. If the PR is unfinished, please mark this PR as draft.
-->
### What changes were proposed in this pull request?
Ensure all events are marked as understorage, this will result to the
LocalStorageMeta being updated when events are processed.
(Please outline the changes and how this PR fixes the issue.)
### Why are the changes needed?
Currently LocalStorageMeta is only update with metrics from the first event
in a given shuffleId and partitionId, the first event updates metrics because
there is no entry in `partitionsOfStorage` and the event get marked as
`underStorage`, however, for future events in the same shuffleId and
partitionId `selectStorage` returns the storage and does not mark the event as
`underStorage` so when `updateWriteMetrics` is called,
`event.getUnderStorage()` returns null and
`storage.updateWriteMetrics(metrics);` is skipped.
As metrics are not updated correctly, `LocalStorage.canWrite` will not
return the correct result.
(Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, describe the bug.) 3.
Fix: # (issue)
### Does this PR introduce _any_ user-facing change?
(Please list the user-facing changes introduced by your change, including
1. Change in user-facing APIs.
2. Addition or removal of property keys.)
No.
### How was this patch tested?
Added unit test which covers multi events for the same shuffleId and
partitionId
(Please test your changes, and provide instructions on how to test it:
1. If you add a feature or fix a bug, add a test to cover your changes.
2. If you fix a flaky test, repeat it for many times to prove it works.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
