[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15532511#comment-15532511
]
Rohith Sharma K S commented on YARN-5585:
-----------------------------------------
I do still think of concern for introducing new field entityPrefixId rather
using createdTime in row key.
# In a distributed cluster, we can expect source of origin of same entity
types from different JVM. For example in MR, what if YarnChild's want to
publish its entities with taskId? How can each yarn child knows about
entityPrefixId? Only uniqueness in cluster will be timestamp.
# If entityPrefixId is string, then expecting user to provide it with padded 0
values like 00001, 00002 etc. It will be a very tedious task for user to decide
what is the length of padding zeros should be used. In long running service,
never able to predict how many number of entities can generated.
If we look at the problem , this issue is from storage layer. To solve this, I
do not feel we need to take solution to the API layer i.e changing
TimelineEntity object. To unblock this issue in API layer, it would be better
to go with API change i.e current patch attached.? And storage layer issue
could be discussed more in other JIRA.
In future, if any other storage is plugged entity prefix would become stale.
> [Atsv2] Add a new filter fromId in REST endpoints
> -------------------------------------------------
>
> Key: YARN-5585
> URL: https://issues.apache.org/jira/browse/YARN-5585
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelinereader
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Priority: Critical
> Attachments: YARN-5585-workaround.patch, YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the
> applications. Along with those, it would be good to add new filter i.e fromId
> so that entities can be retrieved after the fromId.
> Current Behavior : Default limit is set to 100. If there are 1000 entities
> then REST call gives first/last 100 entities. How to retrieve next set of 100
> entities i.e 101 to 200 OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is
> no way to achieve this.
> So proposal is to have fromId in the filter like
> *getApps?limit=5&&fromId=app-5* which gives list of apps from app-6 to
> app-10.
> Since ATS is targeting large number of entities storage, it is very common
> use case to get next set of entities using fromId rather than querying all
> the entites. This is very useful for pagination in web UI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
