[
https://issues.apache.org/jira/browse/YARN-7815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344007#comment-16344007
]
Jason Lowe commented on YARN-7815:
----------------------------------
bq. Would it make sense to detach the appcache and mount a separate appcache
dir for each container? AFAIK it is not for sharing between containers, since
they might get scheduled to other nodes anyways.
It is used for sharing in some circumstances, e.g.: Tez shared fetch where a
task can avoid fetching a broadcast output that another task already fetched,
or Tez local fetch where a downstream task that runs on the same node fetches
an output directly from local disk rather than having it copied through the
shuffle server. Besides those existing use-cases, having a separate appcache
directory per container would add significant load to the shuffle handler,
since it would add another dimension to the search matrix for shuffle data.
Bottom line is we have to mount the application's appcache directory read/write
for backwards compatibility. I don't see that as being a big concern, as
compromising a single container is already compromising the entire application
(due to the application secrets available within that container). The key is
preventing access/corruption to other applications even from the same user.
I think that leaves us with this proposal which should accomplish that and
remove one of the mounts being made today:
1. nm-local-dir/filecache mounted read-only for access to localized public files
2. nm-local-dir/usercache/_user_/filecache mounted read-only for access to
localized user-private files
3. nm-local-dir/usercache/_user_/appcache/_applicationId_ mounted read-write
for access to the application work area and underlying container working
directory
> Mount the filecache as read-only in Docker containers
> -----------------------------------------------------
>
> Key: YARN-7815
> URL: https://issues.apache.org/jira/browse/YARN-7815
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Shane Kumpf
> Assignee: Shane Kumpf
> Priority: Major
>
> Currently, when using the Docker runtime, the filecache directories are
> mounted read-write into the Docker containers. Read write access is not
> necessary. We should make this more restrictive by changing that mount to
> read-only.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
