suvodeep-pyne commented on PR #19909:
URL: https://github.com/apache/pinot/pull/19909#issuecomment-6321284299
@xiangfu0 @raghavyadav01 thanks for the reviews. I pushed a779ff97dd, which
addresses them; I've replied on each thread. The main changes:
- **Re-consuming repairs stay gated.** On pauseless dedup and partial-upsert
tables, the repairs that consume from the stream again are still gated, as on
master: re-ingestion, and the reset of an IN_PROGRESS segment with every
replica in ERROR. The only resets newly enabled there are the ones that reload
a committed segment.
- **Batched, error-only resets.** Resets go through a new
`PinotHelixResourceManager.resetErrorReplicas`, which sends one batch per
instance. It resets only replicas still in ERROR in the instance's current
state, and sends each message from ERROR, so the server drops it if the replica
has already recovered.
- **Gauges.** `PAUSELESS_SEGMENTS_IN_ERROR_COUNT` and
`SEGMENTS_IN_ERROR_STATE` keep their master meaning. The unrecoverable gauge
now counts the segments a pass can't repair or can't issue a repair for, and a
failed reset fails the pass.
The description now covers the behavior changes and upgrade notes. I've also
merged master, including #19739 and #19740, in e06498f41f; CI is green.
On the review-body finding outside the diff
(`RealtimeSegmentValidationManager.processTable`):
- The stale comment is updated.
- On also resetting replicas that are not in ERROR: I've decided not to keep
that, on purpose. A replica whose ExternalView state differs from the
IdealState without being in ERROR is either mid-transition or about to get its
transition from the Helix controller.
- If it is mid-transition, it has a pending message, and
`resetPartitionAllState` refuses a partition with a pending message. On master
that made `resetSegment(table, segment, null)` throw for the segment after the
ERROR replica's reset had gone out. So in your example B was never actually
reset.
- Otherwise, Helix moves B to its IdealState state by itself.
Resetting B mostly risks bouncing a replica that is about to become
healthy, which on dedup and upsert tables drops its keys for a while. This is
listed under the behavior changes in the description.
Could you both take another look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
