[
https://issues.apache.org/jira/browse/YARN-7250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Trezzo updated YARN-7250:
-------------------------------
Description:
We should make the SharedCacheClient use api more consistent with other YARN
api methods. We can do this by doing two things:
# Update the SharedCacheClient#use api so that it returns a URL instead of a
Path. Currently yarn developers have to convert the path to a URL when creating
a LocalResources. It would be much smoother if they could just use a URL passed
to them by the shared cache client.
# Remove the portion of the client that deals with fragments as this is not
consistent with the rest of YARN. This functionality is bleeding in from the
MapReduce layer, which uses fragments to keep track of destination file names.
YARN's api does not use fragments. Instead the ContainerLaunchContext expects
a Map<String, LocalResource> localResources, where the strings are the
destination file names. We should let the YARN application handle destination
file names however it wants instead of pushing this into the shared cache api.
Additionally, fragments are a clunky way to handle this.
was:
We should make the SharedCacheClient use api more consistent with other YARN
api methods. We can do this by doing two things:
# Update the SharedCacheClient#use api so that it returns a URL instead of a
Path. Currently yarn developers have to convert the path to a URL when creating
a LocalResources. It would be much smoother if they could just use a URL passed
to them by the shared cache client.
# Remove the portion of the client that deals with fragments as this is not
consistent with the rest of YARN. This functionality is bleeding in from the
MapReduce layer, which uses fragments to keep track of destination file names.
YARN's api does not use fragments. Instead the ContainerLaunchContext expects
a Map<String, LocalResource> localResources, where the strings are the
destination file names. We should let the YARN application handle destination
file names however it wants instead of pushing this into the shared cache api.
Additionally, fragments is a clunky way to handle this.
> Update Shared cache client api to use URLs
> ------------------------------------------
>
> Key: YARN-7250
> URL: https://issues.apache.org/jira/browse/YARN-7250
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Chris Trezzo
> Assignee: Chris Trezzo
> Priority: Minor
> Attachments: YARN-7250-trunk-001.patch
>
>
> We should make the SharedCacheClient use api more consistent with other YARN
> api methods. We can do this by doing two things:
> # Update the SharedCacheClient#use api so that it returns a URL instead of a
> Path. Currently yarn developers have to convert the path to a URL when
> creating a LocalResources. It would be much smoother if they could just use a
> URL passed to them by the shared cache client.
> # Remove the portion of the client that deals with fragments as this is not
> consistent with the rest of YARN. This functionality is bleeding in from the
> MapReduce layer, which uses fragments to keep track of destination file
> names. YARN's api does not use fragments. Instead the ContainerLaunchContext
> expects a Map<String, LocalResource> localResources, where the strings are
> the destination file names. We should let the YARN application handle
> destination file names however it wants instead of pushing this into the
> shared cache api. Additionally, fragments are a clunky way to handle this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
