[
https://issues.apache.org/jira/browse/HDDS-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated HDDS-13596:
-------------------------------
Description:
This is a ticket to revisit and document listKeys and listStatus consistency
guarantee. Currently, listKeys does not seem to hold any bucket locks and
listStatus seem to only hold keys when retrieving OM key table cache iterator.
During that time, OM DB flush or cache evictions can happen which affect the
list results. Meaning that listKeys currently provides only a "fuzzy" snapshot
of the (similar to ZooKeeper snapshot).
There are concerns raised in HDDS-1986 and HDDS-1987 that seem to be
unaddressed. This includes
* Calling OM DB flush before the list
* Using a generational stamp (shared between cache and DB)
* Holding lock during the list
** This will simplify the consistency but will cause large performance hit
Since our consistency guarantee is read-after-write (or list-after-write in
this case), it might be fine.
was:
This is a ticket to revisit and document listKeys and listStatus consistency
guarantee. Currently, listKeys does not seem to hold any bucket locks and
listStatus seem to only hold keys when retrieving OM key table cache iterator.
During that time, OM DB flush or cache evictions can happen which affect the
list results. Meaning that listKeys currently provides only a "fuzzy" snapshot
of the (similar to ZooKeeper snapshot).
There are concerns raised in HDDS-1986 and HDDS-1987 that seem to be
unaddressed. This includes
* Calling OM DB flush before the list
* Using a generational stamp (shared between cache and DB)
* Holding lock during the list
** This will simplify the consistency but will cause large performance hit
> Revisit ListKeys and ListStatus consistency guarantees
> ------------------------------------------------------
>
> Key: HDDS-13596
> URL: https://issues.apache.org/jira/browse/HDDS-13596
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Ivan Andika
> Priority: Major
>
> This is a ticket to revisit and document listKeys and listStatus consistency
> guarantee. Currently, listKeys does not seem to hold any bucket locks and
> listStatus seem to only hold keys when retrieving OM key table cache
> iterator. During that time, OM DB flush or cache evictions can happen which
> affect the list results. Meaning that listKeys currently provides only a
> "fuzzy" snapshot of the (similar to ZooKeeper snapshot).
> There are concerns raised in HDDS-1986 and HDDS-1987 that seem to be
> unaddressed. This includes
> * Calling OM DB flush before the list
> * Using a generational stamp (shared between cache and DB)
> * Holding lock during the list
> ** This will simplify the consistency but will cause large performance hit
> Since our consistency guarantee is read-after-write (or list-after-write in
> this case), it might be fine.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
