[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-5585:
------------------------------------
Attachment: YARN-5585-YARN-5355.0002.patch
It is quite long time, apologies for this. I worked on this JIRA and tested for
pagination, initial testing for pagination mode looks fine in real cluster.
Attaching incremental patch(but it does not contain REST API changes), but it
still required to modify few code patch based on the discussion.
There are couple of things to discuss and get a consensus
# *REST API definition* : There are set of pre-defined REST API for retrieval
of entities/entity.
*Entities* : I would like to keep same API's for pagination mode. FromId can be
passed as filter where in *fromId=idPrefix!entityId*. User need to input
combination idPrefix and entityId separated by *\!* or *+* symbol in filters.
Reader server parses this combination and start scan from given
idPrefix!entityId row key. Separator to be decided whether *\!* or any other
symbols.
*Entity* : Current entity API's are defined without idPrefix. Should we provide
new API's for entityIdPrefix ? If we plan to reuse same API's, then we need to
handle one scenario where same entityId is published with 2 entityIdPrefix.
entityIdPrefix is mandatorily written even though user do not provide any
idPrefix while publishing entities. So, if case of idPrefix is not known,
should we use default idPrefix to get a row?
# For single entity retrieval, when IdPrefix is not known, need to match column
value for entityType by doing range scan. Any other way this can achieve this?
Thoughts? cc:/ [~gtCarrera9] [~varun_saxena] [~sjlee0] [~vrushalic]
> [Atsv2] Reader side changes for entity prefix and support for pagination via
> additional filters
> -----------------------------------------------------------------------------------------------
>
> Key: YARN-5585
> URL: https://issues.apache.org/jira/browse/YARN-5585
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelinereader
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Priority: Critical
> Labels: oct16-hard
> Attachments: 0001-YARN-5585.patch, YARN-5585-YARN-5355.0001.patch,
> YARN-5585-YARN-5355.0002.patch, YARN-5585-workaround.patch, YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the
> applications. Along with those, it would be good to add new filter i.e fromId
> so that entities can be retrieved after the fromId.
> Current Behavior : Default limit is set to 100. If there are 1000 entities
> then REST call gives first/last 100 entities. How to retrieve next set of 100
> entities i.e 101 to 200 OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is
> no way to achieve this.
> So proposal is to have fromId in the filter like
> *getApps?limit=5&&fromId=app-5* which gives list of apps from app-6 to
> app-10.
> Since ATS is targeting large number of entities storage, it is very common
> use case to get next set of entities using fromId rather than querying all
> the entites. This is very useful for pagination in web UI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
