[
https://issues.apache.org/jira/browse/YUNIKORN-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783497#comment-17783497
]
JiaChi Wang commented on YUNIKORN-2061:
---------------------------------------
Hi [~ccondit] thanks for pointing out the potential problem with pagination. I
think the root cause of this issue is the excessively large response size when
there are numerous applications.
In my test, when there are 500 deployments (each containing 100 pods, totaling
50,000 pods), the response size is approximately 30MB. This doesn't seem
unreasonable, especially in a large cluster. I tested the API using both on
localhost and IP address and found that most of the response time is spent on
data transportation.
So here's another approach to address this problem: we can compress the
response data before sending it, and then decompress the data on the website
side. This approach could minimize changes to the scheduler and address the
issue of excessively large responses.
> [Umbrella] Add Pagination to APIs which may have lot of items in response
> -------------------------------------------------------------------------
>
> Key: YUNIKORN-2061
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2061
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - common, webapp
> Reporter: PoAn Yang
> Assignee: JiaChi Wang
> Priority: Major
> Attachments: Screenshot 2023-10-21 at 8.01.26 PM.png, Screenshot
> 2023-10-21 at 8.03.15 PM.png
>
>
> The following API may have lot of items in one response. It's good to add
> pagination design to them.
> * Queue applications
> (/ws/v1/partition/\{partitionName}/queue/\{queueName}/applications)
> * Application history (/ws/v1/history/apps)
> * Container history (/ws/v1/history/containers)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
