[
https://issues.apache.org/jira/browse/PHOENIX-7978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Gwalani updated PHOENIX-7978:
--------------------------------------
Description:
*Issue Summary*
A _planned_ failover under sustained write load converged *correctly* — no
deadlock, no dual-active, zero RPO, data byte-for-byte identical — but took
{*}~131 s{*}, over the documented *120 s* SLA ceiling. The excess is *not* a
drain-volume problem and *not* the old deadlock. It is a {*}scheduling
artifact{*}: the standby’s replay poller runs on a rigid fixed-rate grid, and
on this run it woke *~20 ms before* the final round became eligible for replay,
so it saw “nothing to do,” went back to sleep, and did not look again for a
{*}full ~60 s cycle{*}. A 20 ms miss cost ~59 s of wall-clock.
*Fix*
**
Two scheduling styles would behave very differently here:
* *Fixed-delay* waits N seconds {_}after each run finishes{_}. A run that
found nothing could be followed by another attempt one interval later — but
more importantly, the phase drifts, so you would not deterministically re-miss
the same eligibility instant.
* *Fixed-rate* (what is used) fires on the immovable grid regardless. When a
tick returns empty there is no short retry — the code just waits for the next
grid slot, ~60 s away.
The poller has no notion of “the thing I’m waiting for will be ready in 20 ms,
let me look again soon.” It looked, saw not-ready, and its next opportunity was
a minute out. That ~59 s of dead time is a pure scheduling artifact: nothing
was draining, nothing was stuck, no data was at risk. The round sat
*eligible-and-unprocessed* from 06:25:09 to 06:26:08 solely because the two
clocks were 20 ms out of phas.
was:TBD
> Avoid delay in Failover due to boundary checks in round start time
> ------------------------------------------------------------------
>
> Key: PHOENIX-7978
> URL: https://issues.apache.org/jira/browse/PHOENIX-7978
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: Himanshu Gwalani
> Assignee: Himanshu Gwalani
> Priority: Major
>
> *Issue Summary*
> A _planned_ failover under sustained write load converged *correctly* — no
> deadlock, no dual-active, zero RPO, data byte-for-byte identical — but took
> {*}~131 s{*}, over the documented *120 s* SLA ceiling. The excess is *not* a
> drain-volume problem and *not* the old deadlock. It is a {*}scheduling
> artifact{*}: the standby’s replay poller runs on a rigid fixed-rate grid, and
> on this run it woke *~20 ms before* the final round became eligible for
> replay, so it saw “nothing to do,” went back to sleep, and did not look again
> for a {*}full ~60 s cycle{*}. A 20 ms miss cost ~59 s of wall-clock.
> *Fix*
> **
> Two scheduling styles would behave very differently here:
> * *Fixed-delay* waits N seconds {_}after each run finishes{_}. A run that
> found nothing could be followed by another attempt one interval later — but
> more importantly, the phase drifts, so you would not deterministically
> re-miss the same eligibility instant.
> * *Fixed-rate* (what is used) fires on the immovable grid regardless. When a
> tick returns empty there is no short retry — the code just waits for the next
> grid slot, ~60 s away.
> The poller has no notion of “the thing I’m waiting for will be ready in 20
> ms, let me look again soon.” It looked, saw not-ready, and its next
> opportunity was a minute out. That ~59 s of dead time is a pure scheduling
> artifact: nothing was draining, nothing was stuck, no data was at risk. The
> round sat *eligible-and-unprocessed* from 06:25:09 to 06:26:08 solely because
> the two clocks were 20 ms out of phas.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
