[
https://issues.apache.org/jira/browse/YARN-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kyungwan nam updated YARN-5696:
-------------------------------
Attachment: YARN-5696-branch-2.7.3.002.patch
add num_containers env variable.
following env variables can be used in shellscript.
NUM_CONTAINERS : a number of containers
CONTAINER_TAG : allocated container_tag. it is between 0 and (NUM_CONTAINERS -
1)
so, input can be divided into num_containers.
for example
{code}
for file in `cat input_files`;
FILE_NUM=`echo $file | cut -d'.' -f 1`
if [ `expr $FILE_NUM % $NUM_CONTAINERS` -eq $CONTAINER_TAG ]; then
echo "$file is assigned to me"
# working with $file
fi
done
{code}
> add container_tag to Distributedshell's env
> -------------------------------------------
>
> Key: YARN-5696
> URL: https://issues.apache.org/jira/browse/YARN-5696
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: applications/distributed-shell
> Reporter: kyungwan nam
> Assignee: kyungwan nam
> Priority: Minor
> Attachments: YARN-5696-branch-2.7.3.001.patch,
> YARN-5696-branch-2.7.3.002.patch
>
>
> a number of containers can be allocated by "num_containers" option.
> but, there is no way to assign different input to each container.
> each container has a unique id between 0 and (num_containers - 1).
> so, user can assign different input (or role) per container.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
