[
https://issues.apache.org/jira/browse/YARN-6345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927330#comment-15927330
]
Vinod Kumar Vavilapalli commented on YARN-6345:
-----------------------------------------------
[~kkaranasos], glad to see movement on this!
I have two broad comments
# We should add allocation-tags instead of container-tags. HBaseMaster can run
in multiple containers over its life-cycle. Yes, today allocation = container.
But that is changing very quickly - ref YARN-4726. Further, the placement
request usually is "place me close to HBaseMaster", not "place me close to this
container-id" - a subtle but important difference.
# API design: We should not add it to existing ResourceRequest object. We have
a vision of moving to a more modular structure for ResourceRequests - YARN-4902
gives a flavor of this.
More context below.
h4. (1) Allocation-tags
Allocation-tags are described in the design doc for YARN-4902.
{code}
Allocation-tags is an option field usable by the scheduler and app developers
to group allocations:
- Every allocation can be associated with a limited number of
allocation-tags. Each allocation-tag is a key-value pair of strings.
- Applications can use an allocation-tag to group and organize pending and
satisfied allocations. For example, a long-running service can assign
different role-types as allocation-tags to different allocations, so that it
can easily understand how much resources are assigned to different roles.
- Applications can also use target-allocation-tags as part of a
placement-strategy described in the later sections. When an application refers
to a target-allocation-tag in the scheduling request, scheduler will use it
to allocate resources for a ResourceRequest respecting affinity/anti-affinity
to another ResourceRequest identified by the passed-in allocation-tags. We
will discuss more details about this in section 4.4 Allocation tags are scoped
under an application (application-id). So, if app A wants to refer to app B’s
allocation-tags as target for placement-strategy (described later), app A
must specify both the application-id of app B as well as the target tags. In
the future, we can see if there is a need to expand this scoping control to be
username, cluster-id etc.
{code}
h4. (2) API design
Today's ResourceRequest is already very messy. We already had to force fit
Allocation ID in semantics and structure (ref YARN-4888) to this. Adding tags
as well as more (YARN-6348) to this already brittle API will lead us to an
evolutionary dead-end. Instead I propose that we start creating a parallel
structure of ResourceRequestCollection proposed in YARN-4902, right under
AllocateRequest record and start bridging from current ResourceRequest object
to the next generation object. We can let them both live and evolve in parallel
by tying the old and new structures via AllocationID.
> Add container tags to resource requests
> ---------------------------------------
>
> Key: YARN-6345
> URL: https://issues.apache.org/jira/browse/YARN-6345
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Konstantinos Karanasos
>
> This JIRA introduces the notion of container tags.
> When an application submits container requests, it is allowed to attach to
> them a set of string tags. The corresponding resource requests will also
> carry these tags.
> For example, a container that will be used for running an HBase Master can be
> marked with the tag "hb-m". Another one belonging to a ZooKeeper application,
> can be marked as "zk".
> Through container tags, we will be able to express constraints that refer to
> containers with the given tags.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
