[
https://issues.apache.org/jira/browse/HDDS-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17929557#comment-17929557
]
Ivan Andika edited comment on HDDS-8633 at 2/26/25 3:50 AM:
------------------------------------------------------------
One possible way is to record the sequence number associated with the OM DB
snapshot downloaded, tails the OM RocksDB WAL starting from that sequence
number and buffer it to a persistent queue (e.g. Chronicle Queue). After the
reprocessing steps are complete, Recon can start consuming the logs from the
persistent queue.
Also we can use bidirectional streaming instead of HTTP downloads.
was (Author: JIRAUSER298977):
One possible way is to record the sequence number associated with the OM DB
snapshot downloaded, tails the OM RocksDB WAL starting from that sequence
number and buffer it to a persistent queue (e.g. Chronicle Queue). After the
reprocessing steps are complete, Recon can start consuming the logs from the
persistent queue.
> Separate Recon OM Synchronization and Tasks Processing
> ------------------------------------------------------
>
> Key: HDDS-8633
> URL: https://issues.apache.org/jira/browse/HDDS-8633
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Recon
> Affects Versions: 1.3.0
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> It is found when Recon is (re)processing all the Recon Tasks (which may take
> quite some time), Recon OM DB updates are blocked. After the Recon finished
> (re)processing tasks, the Recon's last sequence number may have been flushed
> by the OM, causing full snapshot to be triggered again (this can happen
> repeatedly).
> We need to ensure that Recon OM DB is still updated even when Recon Tasks are
> still in (re)processing phase.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
