[
https://issues.apache.org/jira/browse/TEZ-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4358:
------------------------------
Description:
By TEZ-4343, DagClient can expose the sum heap usage by AM and tasks, however,
there were 2 separate API endpoints introduced in order to expose those. This
can become very verbose in code sooner or later once we want to expose more
advanced data. The point is that we don't want to extend the DagClient API
every time when we add a single property that is related to the application's
memory usage. Currently, protobuf doesn't support map, so a workaround is
needed, e.g a prop,value,prop,value serialized list, like:
{code}
list:
["am_consumption",value,"tasks_consumption",value,"task_containers_usage",value1:value2:value3,"task_containers_usage_percentage",value1:value2:value3]
{code}
was:
By TEZ-4343, DagClient can expose the sum heap usage by AM and tasks, however,
there are 2 different API endpoints for that. This can become very verbose
sooner or later when we want to expose more advanced data. The point is that we
don't want to extend the DagClient API every time when we add a single property
that is related to the application's memory usage. Currently, protobuf doesn't
support map, so a workaround is needed, e.g a prop,value,prop,value serialized
list, like:
{code}
list:
["am_consumption",value,"tasks_consumption",value,"task_containers_usage",value1:value2:value3,"task_containers_usage_percentage",value1:value2:value3]
{code}
> Make memory usage provided by DagClient be extensible without further changes
> -----------------------------------------------------------------------------
>
> Key: TEZ-4358
> URL: https://issues.apache.org/jira/browse/TEZ-4358
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
>
> By TEZ-4343, DagClient can expose the sum heap usage by AM and tasks,
> however, there were 2 separate API endpoints introduced in order to expose
> those. This can become very verbose in code sooner or later once we want to
> expose more advanced data. The point is that we don't want to extend the
> DagClient API every time when we add a single property that is related to the
> application's memory usage. Currently, protobuf doesn't support map, so a
> workaround is needed, e.g a prop,value,prop,value serialized list, like:
> {code}
> list:
> ["am_consumption",value,"tasks_consumption",value,"task_containers_usage",value1:value2:value3,"task_containers_usage_percentage",value1:value2:value3]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
