[
https://issues.apache.org/jira/browse/YARN-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15656583#comment-15656583
]
Varun Saxena commented on YARN-5792:
------------------------------------
bq. At a high level, I'd like to discuss options. I see that you're mostly
using the (inverted) start time as the id prefix. Would it be better to use
simply the id instead whenever possible?
I wanted to discuss the same as well. That is why I mixed both approaches in
first patch. I had used Id bit of job id to have id prefix for jobs and attempt
created time for attempts despite it being possible that attempt id can be used.
Anyways in latest patch for YARN and DS attempts I now use the id bit of
attempt id in addition to id bit of job id.
Coming to task IDs', the ID bit of task is cannot really be used because that's
within the scope of task type (map/reduce). Whether task is map or reduce can
be stored in one bit and then we can shift around id. Frankly this should be
fine as one bit of long will be used as sign bit and we do not expect ids' to
be negative. But should we do it. Some mappers may be launched after reducers
are launched so with the approach mentioned above, we may not get that order.
So I would continue going with current approach.
Task attempts are anyways one level below tasks.
Talking about containers, id bit of container ID is a long. And containers are
within scope of an app attempt (id bit of attempt is int). Although number of
attempts are never going to be a very large and id bit of container ID not
reach near the value of long (i.e. not as many containers are likely to be
launched) so we can still adjust both in 8 bytes by doing some bit shifting.
But then we will be making assumptions which IMO we should not do so its safer
to go with container start time. Thoughts ?
> adopt the id prefix for YARN, MR, and DS entities
> -------------------------------------------------
>
> Key: YARN-5792
> URL: https://issues.apache.org/jira/browse/YARN-5792
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-5355
> Reporter: Sangjin Lee
> Assignee: Varun Saxena
> Attachments: YARN-5792-YARN-5355.01.patch,
> YARN-5792-YARN-5355.02.patch, YARN-5792-YARN-5355.03.patch
>
>
> We introduced the entity id prefix to support flexible entity sorting
> (YARN-5715). We should adopt the id prefix for YARN entities, MR entities,
> and DS entities to take advantage of the id prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
