[
https://issues.apache.org/jira/browse/YARN-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099207#comment-16099207
]
Vrushali C edited comment on YARN-6733 at 7/24/17 11:28 PM:
------------------------------------------------------------
So we thought that it will be good to keep the column name so that sub apps can
store this information. For regular applications, the flow version can be used
to determine whether optimizations are to be done. The flow version indicates
if the flow has changed, that is, say if the pig script changes, its flow
version will change. So then, for example, reducer estimation calculations can
be done differently. This applies to the application entities. We discussed
that it will be good to keep the same information for sub-apps in case they
want to use this information in a similar fashion. As such, this column
currently only exists in code, it's not taking up any disk space/hbase space
etc if no one writes to it. But having it gives the framework developers a
chance to use it if they want.
was (Author: vrushalic):
So we thought that it will be good to keep the column name so that sub apps can
store this information. For regular applications, the flow version can be used
to determine whether optimizations are to be done. The flow version indicates
if the flow has changed, that is, say if the pig script changes, it's flow
version will change. So then, for example, reducer estimation calculations can
be done differently. This applies to the application entities. We discussed
that it will be good to keep the same information for sub-apps in case they
want to use this information in a similar fashion. As such, this column
currently only exists in code, it's not taking up any disk space/hbase space
etc if no one writes to it. But having it given the framework developers a
chance to use it if they want.
> Add table for storing sub-application entities
> ----------------------------------------------
>
> Key: YARN-6733
> URL: https://issues.apache.org/jira/browse/YARN-6733
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Vrushali C
> Assignee: Vrushali C
> Attachments: IMG_7040.JPG, YARN-6733-YARN-5355.001.patch,
> YARN-6733-YARN-5355.002.patch, YARN-6733-YARN-5355.003.patch,
> YARN-6733-YARN-5355.004.patch, YARN-6733-YARN-5355.005.patch,
> YARN-6733-YARN-5355.006.patch, YARN-6733-YARN-5355.007.patch,
> YARN-6733-YARN-5355.008.patch
>
>
> After a discussion with Tez folks, we have been thinking over introducing a
> table to store sub-application information.
> For example, if a Tez session runs for a certain period as User X and runs a
> few AMs. These AMs accept DAGs from other users. Tez will execute these dags
> with a doAs user. ATSv2 should store this information in a new table perhaps
> called as "sub_application" table.
> This jira tracks the code changes needed for table schema creation.
> I will file other jiras for writing to that table, updating the user name
> fields to include sub-application user etc.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
