[
https://issues.apache.org/jira/browse/HADOOP-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Craig Macdonald resolved HADOOP-2862.
-------------------------------------
Resolution: Duplicate
Fix Version/s: 0.19.0
> [HOD] Support PBS env vars in hod configuration
> -----------------------------------------------
>
> Key: HADOOP-2862
> URL: https://issues.apache.org/jira/browse/HADOOP-2862
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/hod
> Affects Versions: 0.16.0
> Environment: Torque PBS
> Reporter: Craig Macdonald
> Fix For: 0.19.0
>
>
> In some batch environments, eg using Torque PBS, scratch spaces are provided
> on cluster nodes for where jobs should put their temporary files. These are
> automatically cleaned up when the job exists by an epilogue script.
> For instance, in our local Torque cluster, all nodes have a /scratch
> partition. For each job, the prologue script creates a scratch folder owned
> by the user at /scratch/pbstmp.$PBS_JOBID - $PBS_JOBID is then the env var
> containing the job id, as set by pbs_mom.
> Would it be possible to use these env vars in the configuration of hod. For
> instance, say I want to create an hdfs on demand using hod, but that the hdfs
> space should be in /scratch/pbstmp.$PBS_JOBID, not in /tmp/hod say. This
> would involve HOD supporting env vars in configuration, but knowing when to
> substitute the env var with it's current value (ie not until running on the
> correct node where the operation should take place).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
