dombizita commented on PR #5658: URL: https://github.com/apache/ozone/pull/5658#issuecomment-1924117871
While working and reviewing this patch we found that the pagination on the OM DB Insights page is not working correctly at all. @smitajoshi12 tried to fix it this PR, but it seems that part needs a bigger change, so we agreed to do that in another jira. I filed [HDDS-10282](https://issues.apache.org/jira/browse/HDDS-10282) to do that. The problem is that currently all the requests (for all 4 endpoints) have both the `limit` and the `prevKey` parameters set. The `limit` is set based on the pages size and the `prevKey` based on the previous page's last element. Another thing we should notice is that we can set the page size (10, 20, 30, 50). To be able to navigate back we would need to know the last element of the previous page's previous page. To solve this @smitajoshi12 created a map in this PR, where we are storing the page numbers as keys and the last elements of the previous pages as values paired to it. After you put a key-value pair to the map, they won't change the value later. The problems we faced because of this: - If we are changing the size of the page, we will read the `prevKey` from the map, which will be the same that was already calculated for a different page size; only the limit will be changed in the request. We will face a problem, where we are showing the same elements on consecutive pages. I described one scenario here: https://github.com/apache/ozone/pull/5658#discussion_r1469353863 - On each page we are only loading the page size number of elements, so all the sorting and searching (based on the columns) will happen only with the elements on the page. I looked into what kind of solutions are there at other UI platforms. First of all a problem that we are facing, is that we have key-value store [table interface](https://github.com/apache/ozone/blob/4b9b10f6206ed5c29c0c4957a651de1d24fe86f6/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/Table.java), so we can't get an element based on offset. Another solution could be if we won't let users to change the page size and use the map solution, but I don't like the idea that the search/sorting is not working on the whole dataset that we have requested. I'd suggest to follow the same solution that @ivandika3 did in #4585, so we would retrieve a limited amount of the data, that can be set via the user and the whole pagination is solved by the ant library. Later we can add an interface in Recon, where you can ask for a specific key/container with the APIs present on the OM DB Insights page. What do you think of this approach? @smitajoshi12 @ArafatKhan2198 @devmadhuu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
