[
https://issues.apache.org/jira/browse/YARN-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15779726#comment-15779726
]
Varun Saxena commented on YARN-6027:
------------------------------------
[~sunilg]
bq. but if some one want to see all flows, i still feel we need a landing page
Currently this is possible but a full table scan will be required. And
furthermore, either client side processing or another REST endpoint would be
required to just give a list of distinct flows instead of activities. And
pagination for a set of flows will not be feasible with the current table
schema.
Its not 24 hour listing only but listing based on dates.
Now coming to the intention behind current table schema.
Assume the list of flows, runs into thousands. How do we know which is the
latest flow then ? Also each flow can have multiple flow runs i.e. same flow
can be run multiple times. For instance, take a set of queries executed
periodically. Unlike application ID, we cannot order the flows by flow name.
A user is more likely to query a recent flow than look at something which was
active a month ago. So ordering it by day timestamp, we thought was reasonable.
We wanted to keep this table light hence we make an entry to it only when an
application begins or finishes. Please note that there is no concept of flow
beginning or finishing.
Frankly, flows in ATSv2 are meant to give admins/users an easy way to access
what they have executed. An end user is more likely to know about a flow name
than multiple application IDs' inside a flow. How about we provide a way to
search for a flow name in UI and list down all its activities, if he cannot
find it amongst recent activities ? A flowname query param can be provided to
make this search feasible from timeline service side. Thoughts ?
> Support fromId for flows/flowrun apps
> -------------------------------------
>
> Key: YARN-6027
> URL: https://issues.apache.org/jira/browse/YARN-6027
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Labels: yarn-5355-merge-blocker
>
> In YARN-5585 , fromId is supported for retrieving entities. We need similar
> filter for flows/flowRun apps and flow run and flow as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
