Hi Clément,
The HADOOP_USER_NAME trick will work only in a non-secure cluster where you
can be whoever you want just by setting that property. it will not work in
a secure setup.
The JavaAction has the same limitation as the shell action (the shell
action is a specialized java action that just launches a shell).
Said this, would you try the following? passing the following option to all
your hadoop tool invocations from your shell
-Dmapreduce.job.credentials.binary=${HADOOP_TOKEN_FILE_LOCATION}
If this work, it will work for both unsecure and secure clusters.
Thx.
On Thu, Feb 21, 2013 at 9:22 AM, Clément MATHIEU <[email protected]>wrote:
> On 2013-02-21 18:01, Alejandro Abdelnur wrote:
>
> Thanks Alejandro,
>
> Oozie provides MR and Pig actions that propagate the user identity
>> correctly. If you are using a shell action your shell action ends up
>> running as the Unix user running the task, in an unsecure cluster is
>> typically the mapred user, in a secure cluster is the same user that
>> submitted the job to Oozie
>>
>
> I fully understand that. My question was about how to make things work for
> the shell action since I believe that my use case is legitimate. What is
> the purpose of the shell action if even a simple operation like creating a
> new file leads to a bad file ownership ?
>
> you want to run jobs against the cluster using the correct user, you
>> should
>> use the Oozie provided actions for that effect (MR, Pig, Sqoop, Hive,
>> distcp, etc).
>>
>
> In the meantime I did some experimentation with the HADOOP_USER_NAME
> environment variable. Specifying it solve almost all my issues (Pig & MR
> job are run using the right user, files have the right owner). You even
> don't have to hard code the user name since you can use the user() EL
> function.
>
> <exec>[SHELL-COMMAND]</exec>
> <argument>[ARG-VALUE]</**argument>
> ...
> <env-var>HADOOP_USER_NAME=${**wf:user()}</env-var>
> <file>[FILE-PATH]</file>
> ...
> </exec>
>
>
> Wouldn't it be good to do something like this automatically rather the put
> the burden on the user ? I don't see why an user would prefer the current
> behavior.
>
> I'm not sure if this solution would work in secure mode too. But I
> definitely believe that there is a better option than forcing the user to
> always use Oozie actions even when another tool is more suitable for the
> job.
>
>
> BTW: Does the Java action suffers from the same limitation ?
>
>
> Thanks,
>
>
> -- Clément
>
--
Alejandro
