[
https://issues.apache.org/jira/browse/YARN-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340325#comment-16340325
]
Vrushali C commented on YARN-3895:
----------------------------------
Hi [~jlowe]
We discussed this between [~rohithsharma], [~lohit], [~varun_saxena] and I. It
basically comes down to whether we want to take a performance hit at read time
or write time. Given that writing out extra details at write time seems like
the worse option when running at scale, we thought of taking the approach
which may be a slight hit on the read path but has some optimizations.
Here is our proposal.
Extremely short summary:
We will go with the domain concept that comes with ATSv1. So each entity is
written with a domain id. At read time, the check is made to ensure the
querying user has permissions to read the data based on domain id.
Design Details:
Domain ID storage:
- domains are published by the AM, just as they are done in ATSv1.
- subsequent entity writes include the domain id per write, same as ATSv1.
- domain ids are written to two tables in hbase.
- one table is user_domain table and the other is groups_domain table.
- the user_domain table has the rowkey as cluster id + username and a column
whose value is the list of domain ids for that user.
- Similarly the groups_domain table rowkey of cluster id + group name and a
column whose value stores the list of domain ids for that group.
So, for each user or group in the timeline domain object who is a reader or the
owner, the domain id is added to that user's row in the user_domain or
groups_domain table. The domain id is first written to the cell with tags. Now,
there will be a coprocessor which checks if the domain id already exists in the
value in the domain column. If yes, no-op, nothing to do. If the domain id does
not already exist, meaning it is a new one, it will be appended to the value
list.
- Expiration/ removal of domain ids.
If this list of domain ids has the potential to grow very big, we can consider
storing a TTL for each domain id. We can store the TTLs per domain id in these
user_domain and group_domain tables and have the coprocessor look at cleanup at
the time of major compaction.
If the list of domain ids is small enough, expiration / TTL is not required to
be implemented. What do you think? How many domains would there be?
Read Query time:
We propose to have the reader api authorization to work in the following
fashion.
- A read query for an entity comes in from a user.
- The timeline reader will create 3 threads and issue three parallel requests
to hbase.
- One request is a Get from the user_domain table for this querying user. Gets
back a list of domain ids this user has permissions for.
- Another request is a Get from the groups_domain table for the group that
this querying user belongs to. Gets back a list of domain ids this group has
permissions for. This may be pretty big?
- Third request is to get the entities that are being asked for .
Now, given the domain ids in the entity response, a check is made if the domain
id exists in the user_domain response or the groups_domain response.
This dataset is accordingly returned as the query response. I believe ATSv1
does a get all entities and then queries the domain table to see if this domain
id relates to that querying user. This model may not work efficiently in hbase
in case of multiple domain ids, doing too many gets will make the timeline
reader response slow.
But, as an additional api option, if the domain id is passed into the query, we
can check for existence of that domain id directly in the user_domain or
groups_domain table and proceed accordingly.
Also, if the user who is querying is an admin user, we can skip all the checks
and just get the entities. And of course, if security is not enabled, no
additional gets from user_domain and groups_domain table are required.
What do you think of this approach?
> Support ACLs in ATSv2
> ---------------------
>
> Key: YARN-3895
> URL: https://issues.apache.org/jira/browse/YARN-3895
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Priority: Major
> Labels: YARN-5355
>
> This JIRA is to keep track of authorization support design discussions for
> both readers and collectors.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
