Can you please send the output snippets of $hadoop fs -lsr /user/oozie/share/
On Wednesday, February 11, 2015 2:06 AM, Jakub Stransky
<[email protected]> wrote:
Hello experienced users,
anybody faced similar issue? I am stack on this for 3 day and still not
found what might be a problem. Since the installation of share libs is
correct as I see a corect folder with a timestamp but cannot figure it out
why this is not added to classpath even when I
specify oozie.use.system.libpath=true in job.properties. When tried
strace oozie
admin -shareliblist command I can see a failing child process but can not
nail it down.
Does anybody has some suggestion what to try or check?
On 6 February 2015 at 10:44, Jakub Stransky <[email protected]> wrote:
> Hello,
>
> I tried a new installation procedure for shreLib installation after
> overcoming issues like: user oozie cannot impersonate user oozie etc. I got
> share/libs sucessfully installed to HDFS under
> /user/oozie/share/lib/lib_20141107094835 which I guess is fine. BUT the
> result of the command
> oozie admin -shareliblist
>
> returns null
>
> The commad for sharelib creation doesn't print any error to output nor
> oozie servers logs are free of any error. Can somebody help me to nail it
> down? Where oozie keep the information which share lib should be used?
>
> Thanks for helping me out
>
> Jakub
>
>
> On 4 February 2015 at 23:27, Jakub Stransky <[email protected]> wrote:
>
>> No, I didn't I used instruction from apache oozie sites 4.1.0 which are
>> not different from the old one. WIll give a try!
>>
>> Thanks for pointing this out
>> Jakub
>>
>>
>> On 4 February 2015 at 22:55, Mohammad Islam <[email protected]>
>> wrote:
>>
>>> Hi Jakub,Shared library structure was changed in 4.1.0.
>>> It is now in /user/oozie/shared/lib/lib_$timestamp.
>>> Did you follow the new instructions?Some discussion at:How-to: Use the
>>> ShareLib in Apache Oozie (CDH 5)
>>>
>>> | |
>>> | | | | | |
>>> | How-to: Use the ShareLib in Apache Oozie (CDH 5)drwxr-xr-x
>>> share/lib/lib_20140403151601/distcp -rw-r--r--
>>> share/lib/lib_20140403151601/distcp/hadoop-distcp-2.3.0-cdh5.0.0.jar |
>>> | |
>>> | View on blog.cloudera.com | Preview by Yahoo |
>>> | |
>>> | |
>>>
>>>
>>> Regards,Mohamamd
>>>
>>>
>>>
>>>
>>> On Wednesday, February 4, 2015 7:03 AM, Jakub Stransky <
>>> [email protected]> wrote:
>>>
>>>
>>> Hello experienced oozie users,
>>>
>>> we have upgraded our infrastructure from hdp 2.0 to hdp 2.2 which means
>>> upgrade from hadoop 2.2.0 to 2.6.0 and oozie from 4.0.0 to 4.1.0. We are
>>> using simple authentication etc. When the job gets submitted oozie
>>> launcher
>>> is failing with following:
>>>
>>> 2015-02-04 09:56:33,364 WARN [main]
>>> org.apache.hadoop.mapred.YarnChild: Exception running child :
>>> java.io.IOException: Split class
>>> org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not
>>> found
>>> at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363)
>>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:423)
>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>>> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at javax.security.auth.Subject.doAs(Subject.java:415)
>>> at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>>> Caused by: java.lang.ClassNotFoundException: Class
>>> org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not
>>> found
>>> at
>>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1982)
>>> at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361)
>>> ... 7 more
>>>
>>> 2015-02-04 09:56:33,373 INFO [main] org.apache.hadoop.mapred.Task:
>>> Runnning cleanup for the task
>>>
>>> Found that this classes are contained in oozie share/lib uploaded to HDFS
>>> to /user/${user.name}/share/lib according to oozie-site.xml
>>> configuration:
>>>
>>> <property>
>>> <name>oozie.service.WorkflowAppService.system.libpath</name>
>>> <value>/user/${user.name}/share/lib</value>
>>> </property>
>>>
>>> where ${user.name} is user under which oozie server is running e.g.
>>> oozie.
>>> For some reason those libs doesn't get propagated to
>>> mapreduce.application.classpath and hence ClassNotFoundException.
>>>
>>> Tried to change following property to true:
>>>
>>> <property>
>>> <name>use.system.libpath.for.mapreduce.and.pig.jobs</name>
>>> <value>true</value>
>>> </property>
>>>
>>> But with no difference. I am not sure what I am missing in installation
>>> instruction here
>>> <
>>> http://oozie.apache.org/docs/4.1.0/DG_QuickStart.html#Oozie_Share_Lib_Installation
>>> >
>>>
>>> Any hint would be appreciated.
>>>
>>> Thx
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Jakub Stransky
>> cz.linkedin.com/in/jakubstransky
>>
>>
>
>
> --
> Jakub Stransky
> cz.linkedin.com/in/jakubstransky
>
>
--
Jakub Stransky
cz.linkedin.com/in/jakubstransky
