[
https://issues.apache.org/jira/browse/HDDS-13365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated HDDS-13365:
-------------------------------
Description:
We need to ensure that the tables in CleanupTableInfo in OM response should be
exactly the same as the ones updated in the corresponding OM request's
validateAndUpdateCache. Currently, this is done with using CleanupTableInfo
annotations on the OMClientResponse. However, this is hard to keep track
because all the cache updates are done in the OMClientRequest, but we need to
update the OMClientResponse instead, which is very error-prone. We should
couple the table cache update in OMClientRequest and the tables to cleanup.
That way, we remove the CleanupTableInfo reflection entirely. This also have
added benefit of reducing the overhead introduced by Reflections
([https://stackoverflow.com/questions/435553/java-reflection-performance]),
this requires some benchmark to ensure there are no performance regressions.
One way is for the OMClientRequest to pass the table cache to cleanup for the
OmClientResponse whenever a cache is updated (e.g. using some context object).
This will ensure that any table cache updated will be propagated to the
OMClientResponse and OMClientResponse do not need to handle this logic.
Moreover, this can save some cleanup overhead if not all tables in the
CleanupTableInfo need to be cleaned up in certain cases (e.g.
OMOpenKeysDeleteRequest might only need to cleanup only OPEN_KEY_TABLE (if all
the open keys are from OBS / LEGACY buckets) or OPEN_FILE_TABLE (if all the
open keys are from FSO bucket) or both. For example, for each
Table#addCacheEntry, we also need to update the list of tables to cleanup.
However, there might be some memory overhead when passing the OM table cache to
cleanup (although the context objects should be short-lived).
Currently, since OMClientRequest#validateAndUpdateCache only returns the
generic OMClientResponse, it might be hard to test this. We might need to
parameterize ( the OMClientRequest (using generic) to pair it with the
associated OMClientResponse implementation. Afterwards, we can list all the
OMClientRequest and check the associated OMClientResponse.
was:
We need to ensure that the tables in CleanupTableInfo in OM response should be
exactly the same as the ones updated in the corresponding OM request's
validateAndUpdateCache. Currently, this is done with using CleanupTableInfo
annotations on the OMClientResponse. However, this is hard to keep track
because all the cache updates are done in the OMClientRequest, but we need to
update the OMClientResponse instead, which is very error-prone. We should
couple the table cache update in OMClientRequest and the tables to cleanup.
That way, we remove the CleanupTableInfo reflection entirely. This also have
added benefit of reducing the overhead introduced by Reflections
([https://stackoverflow.com/questions/435553/java-reflection-performance]).
One way is for the OMClientRequest to pass the table cache to cleanup for the
OmClientResponse whenever a cache is updated (e.g. using some context object).
This will ensure that any table cache updated will be propagated to the
OMClientResponse and OMClientResponse do not need to handle this logic.
Moreover, this can save some cleanup overhead if not all tables in the
CleanupTableInfo need to be cleaned up in certain cases (e.g.
OMOpenKeysDeleteRequest might only need to cleanup only OPEN_KEY_TABLE (if all
the open keys are from OBS / LEGACY buckets) or OPEN_FILE_TABLE (if all the
open keys are from FSO bucket) or both. For example, for each
Table#addCacheEntry, we also need to update the list of tables to cleanup.
However, there might be some memory overhead when passing the OM table cache to
cleanup (although the context objects should be short-lived).
Currently, since OMClientRequest#validateAndUpdateCache only returns the
generic OMClientResponse, it might be hard to test this. We might need to
parameterize ( the OMClientRequest (using generic) to pair it with the
associated OMClientResponse implementation. Afterwards, we can list all the
OMClientRequest and check the associated OMClientResponse.
> Ensure that CleanupTableInfo in OM response is synced with OM request
> ---------------------------------------------------------------------
>
> Key: HDDS-13365
> URL: https://issues.apache.org/jira/browse/HDDS-13365
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> We need to ensure that the tables in CleanupTableInfo in OM response should
> be exactly the same as the ones updated in the corresponding OM request's
> validateAndUpdateCache. Currently, this is done with using CleanupTableInfo
> annotations on the OMClientResponse. However, this is hard to keep track
> because all the cache updates are done in the OMClientRequest, but we need to
> update the OMClientResponse instead, which is very error-prone. We should
> couple the table cache update in OMClientRequest and the tables to cleanup.
> That way, we remove the CleanupTableInfo reflection entirely. This also have
> added benefit of reducing the overhead introduced by Reflections
> ([https://stackoverflow.com/questions/435553/java-reflection-performance]),
> this requires some benchmark to ensure there are no performance regressions.
> One way is for the OMClientRequest to pass the table cache to cleanup for the
> OmClientResponse whenever a cache is updated (e.g. using some context
> object). This will ensure that any table cache updated will be propagated to
> the OMClientResponse and OMClientResponse do not need to handle this logic.
> Moreover, this can save some cleanup overhead if not all tables in the
> CleanupTableInfo need to be cleaned up in certain cases (e.g.
> OMOpenKeysDeleteRequest might only need to cleanup only OPEN_KEY_TABLE (if
> all the open keys are from OBS / LEGACY buckets) or OPEN_FILE_TABLE (if all
> the open keys are from FSO bucket) or both. For example, for each
> Table#addCacheEntry, we also need to update the list of tables to cleanup.
> However, there might be some memory overhead when passing the OM table cache
> to cleanup (although the context objects should be short-lived).
> Currently, since OMClientRequest#validateAndUpdateCache only returns the
> generic OMClientResponse, it might be hard to test this. We might need to
> parameterize ( the OMClientRequest (using generic) to pair it with the
> associated OMClientResponse implementation. Afterwards, we can list all the
> OMClientRequest and check the associated OMClientResponse.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For additional commands, e-mail: issues-h...@ozone.apache.org
