AmatyaAvadhanula opened a new pull request, #16393:
URL: https://github.com/apache/druid/pull/16393
```
SegmentTransactionalReplaceAction
0) Validate replace segments with locks
1) Allocate ids for segments to upgrade for replace intervals
2) Commit segments to replace and upgraded segment ids
3) Upgrade pending segments overlapping with segments
SegmentTransactionalAppendAction
0) Validate append segments with locks
1) Fetch all upgraded pending segments corresponding to segments to be
appended
2) Create upgrade entries for append segments for which there are active
replace locks
3) Delete pending segment entries for segments to be appended and their
upgraded versions
4) Commit segments to be appended with their upgraded versions
allocate and append X0
replace Y1
1)
R1 - noop
R2 - commit Y1
R3 - upgrade X0 to X1
A1 - fetch X0 X1
A2 - noop
A3 - delete X0, X1 pending segments
A4 - commmit X0, X1
Result - as expected
1)
R1 - noop
R2 - commit Y1
A1 - fetch X0
R3 - upgrade X0 to X1
A2 - noop
A3 - delete X0,
A4 - commmit X0
Result - failure
...
```
There may be other interleavings of the calls in these actions that may lead
to incorrect states.
The solution is to use read / write locks for segment transactional append /
replace actions respectively.
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency
checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md)
(Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] a release note entry in the PR description.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked
related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in
[licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
