[
https://issues.apache.org/jira/browse/HDDS-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685044#comment-17685044
]
Devesh Kumar Singh commented on HDDS-7465:
------------------------------------------
#HDDS-7468 has resolved this issue.
> Recon - {containerId}/keys API is not giving details of keys for missing
> containers when recon was down
> -------------------------------------------------------------------------------------------------------
>
> Key: HDDS-7465
> URL: https://issues.apache.org/jira/browse/HDDS-7465
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Devesh Kumar Singh
> Assignee: Devesh Kumar Singh
> Priority: Major
> Attachments: image-2022-11-09-09-34-16-930.png
>
>
> Recon - \{containerId}/keys API is not giving details of keys for missing
> containers when recon was down.
>
> Steps to reproduce:
> # Create a 3 data node cluster and non-HA single SCM cluster
> # Bring Recon down.
> # Create few volumes and buckets and keys to have few (say 5-6) containers
> created.
> # SCM rocks DB got update of these containers. Make sure that
> "ozone.scm.ratis.enable" property is set as false to immediate flush of
> containers and their state in SCM rocks DB.
> 5. Bring all the 3 the data nodes down.
> 6. Containers will move from OPEN to CLOSING. Bring all the 3 DNs up
> and make sure that containers state moved from QUASI CLOSED to CLOSED.
> 7. Bring all the 3 data nodes down again.
> 8. Bring Recon up and all containers should be reported as missing.
> 9. For all missing containers, expand container id to list the key and
> volume info.
> Expected: Key and volume info should be listed for missing container id.
> Actual: Key and volume is not listed for missing container id.
>
> Root cause: When Recon was down, recon doesn't get any updates (PUT, UPDATE,
> DELETE) for keys created from OM, so no delta updates of OM gets applied for
> Recon OM DB and containerKey mapping data is empty.
>
> Solution: Get full periodic snapshot of OM rocks db and and then process the
> data to get containerKeyMapping info because if Recon was down when
> containers gets created, full snapshot of OM DB will never gets taken due to
> below logic as this logic relies on currentSequenceNumber which will never be
> less than equal to zero as OM DB sequence number has already moved beyond
> zero when keys were written.
>
> !image-2022-11-09-09-34-16-930.png!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
