[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15633179#comment-15633179
]
Varun Saxena commented on YARN-5585:
------------------------------------
Thanks [~rohithsharma] for the patch. Will touch upon the areas which have been
completed.
# In GenericEntityReader#getResults we need to have a PageFilter based on
TimelineEntityFilters#limit.
# You commented about the GenericEntityReader#getResults method. I agree that
we can have {{Long}} in EntityRowKey to get the prefix which can then be used
to set stop row. Current code has a problem as you mentioned. We should have
now either 2 filters fromPrefix and fromId or a single filter which carries
both separated by some separator. Right ?
# Should we move calculateTheClosestNextRowKeyForPrefix to TimelineStorageUtils
?
# Javadoc in TimelineReader should be changed. It currently says entities would
be sorted by created time which is no longer true.
{code}
* @return A set of <cite>TimelineEntity</cite> instances of the given entity
* type in the given context scope which matches the given predicates
* ordered by created time, descending. Each entity will only contain the
* metadata(id, type and created time) plus the given fields to retrieve.
{code}
# Regarding below piece of code in GenericEntityReader#getResults. I think to
differentiate between pagination and non pagination mode we should merely use
the new filters we add instead checking the context. These filter(s) should go
into TimelineEntityFilters (semantically speaking) instead of going in
TimelineReaderContext i.e. we should not be filling entity ID and prefix in
context based on fromId + fromPrefix filter.
{code}
if (context.getEntityId() == null) {
...
} else { // pagination mode, will scan from given entityPrefix!enitityId
...
}
{code}
> [Atsv2] Reader side changes for entity prefix and support for pagination via
> additional filters
> -----------------------------------------------------------------------------------------------
>
> Key: YARN-5585
> URL: https://issues.apache.org/jira/browse/YARN-5585
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelinereader
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Priority: Critical
> Labels: oct16-hard
> Attachments: 0001-YARN-5585.patch, YARN-5585-YARN-5355.0001.patch,
> YARN-5585-workaround.patch, YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the
> applications. Along with those, it would be good to add new filter i.e fromId
> so that entities can be retrieved after the fromId.
> Current Behavior : Default limit is set to 100. If there are 1000 entities
> then REST call gives first/last 100 entities. How to retrieve next set of 100
> entities i.e 101 to 200 OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is
> no way to achieve this.
> So proposal is to have fromId in the filter like
> *getApps?limit=5&&fromId=app-5* which gives list of apps from app-6 to
> app-10.
> Since ATS is targeting large number of entities storage, it is very common
> use case to get next set of entities using fromId rather than querying all
> the entites. This is very useful for pagination in web UI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
