Hi Ivan,
Thanks for the explanation!
> "draining" state is not something that anyone but the auditor needs to
> care about. It's not a state attribute of the bookie,
> but instead an external service's opinion of what should be happening
> with the bookie. as such, it doesn't belong in the bookie service info
> itself.
>
> It should go in some metadata that the auditor is maintaining about
> the bookie, so that when the auditor restarts, it can see that it
> should prioritize
> moving data from that bookie.
This API is a bit of a hodge podge of different kinds of state.
> "shutting down", "running" and "availableForHighPriority" are
> transient states. We need read-only to be persistent if it is
> requested by an external entity. To persist the readonly state, there
> are multiple options. We can persist it to zookeeper somehow, or we
> can persist it as file on the bookie's disk. But the API linked isn't
> really suited in any case.
> What we need is a /bookie/state/readonly endpoint, where we PUT and
> payload like '{"readonly": true}'. When this gets a OK response, that
> state should be persisted, so any reboot will keep the bookie in the
correct state.
Besides the auditor, I think the external operator (whether a human
operator or an automation program) also cares about the "draining" state of
a bookie. Assuming we have all the features discussed in this thread added
to the bookkeeper, before removing a bookie from the cluster, if the data
is expected to be offloaded to the tiered storage or just wait for the
retention period to pass, it's sufficient to set the bookie to be readonly.
If the data is expected to be moved off the bookie by auto-recovery, the
bookie has to be set as "draining" to kick off the data migration, and
there should also be APIs to mark a bookie as "draining" and to check if
the bookie is in the "draining" state. Although "draining" is a special
case of "readonly", would it be more clear to make it another possible
value of `BookieMode` and provide similar APIs as for the readonly state?
Or do you have any suggestions on the management of the draining state and
relative APIs?
Best regards,
Yang Yang
