This seems not to be working.
There is a JIRA marked resolved but that is really not a FIXED. https://issues.apache.org/jira/browse/OOZIE-2066 I even tried various combo of : mapreduce.task.classpath.user.precedence mapreduce.task.classpath.first mapreduce.job.user.classpath.first oozie.launcher.mapreduce.task.classpath.user.precedence My Java Action needs httpclient-4.3.1.jar and httpcore-4.3.1.jar files. But somehow these are overridden by : /opt/hes/hadoop/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar /opt/hes/hadoop/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar In the Hadoop logs I see that in java.class.path the ../ hadoop/common/lib/httpclient-4.2.5.jar and ../ hadoop/common/lib/httpclient-4.2.5.jar are preceded with my httpclient-4.3.1.jar and httpcore-4.3.1.jar. The error in Hadoop Logs: <<< Invocation of Main class completed <<< Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main() threw exception, java.lang.NoSuchFieldError: INSTANCE org.apache.oozie.action.hadoop.JavaMainException: java.lang.NoSuchFieldError: INSTANCE at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:59) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47) at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:35) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.NoSuchFieldError: INSTANCE at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<init>(DefaultHttpRequestWriterFactory.java:52) at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<init>(DefaultHttpRequestWriterFactory.java:56) at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<clinit>(DefaultHttpRequestWriterFactory.java:46) at org.apache.http.impl.conn.ManagedHttpClientConnectionFactory.<init>(ManagedHttpClientConnectionFactory.java:72) at org.apache.http.impl.conn.ManagedHttpClientConnectionFactory.<init>(ManagedHttpClientConnectionFactory.java:84) at org.apache.http.impl.conn.ManagedHttpClientConnectionFactory.<clinit>(ManagedHttpClientConnectionFactory.java:59) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$InternalConnectionFactory.<init>(PoolingHttpClientConnectionManager.java:487) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:147) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:136) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:112) at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:727) Can anyone has a solution to override Hadoop native libs with the user defined jars in workflow/lib folder of your Oozie app? -Nirmal -----Original Message----- From: Sabbidi, Prashanth [mailto:[email protected]] Sent: Friday, August 21, 2015 11:26 PM To: [email protected]; Purshotam Shah <[email protected]> Subject: RE: Override single jar in Oozie hsared lib Thankyou Jay, I'll try this option. -----Original Message----- From: Jaydeep Vishwakarma [mailto:[email protected]] Sent: Friday, August 21, 2015 12:53 PM To: [email protected]<mailto:[email protected]>; Purshotam Shah Subject: Re: Override single jar in Oozie hsared lib I can suggest you some workaround for this. Can you put joda2.6 jar in the lib directory of workflow and add the following property in pig action. <property> <name>oozie.launcher.mapreduce.task.classpath.user.precedence</name> <value>true</value> </property> I have tested this with hadoop 2.6. Regards, Jaydeep On Fri, Aug 21, 2015 at 9:56 PM, Purshotam Shah < [email protected]<mailto:[email protected]>> wrote: > Unfortunately it's not possible now. We are working on it and may be > available in next release.https://issues.apache.org/jira/browse/OOZIE-1624 > From: "Sabbidi, Prashanth" > <[email protected]<mailto:[email protected]>> > To: "[email protected]<mailto:[email protected]>" > <[email protected]<mailto:[email protected]>> > Sent: Friday, August 21, 2015 8:47 AM > Subject: Override single jar in Oozie hsared lib > > Hi, > > Can we override single jar in Oozie shared lib, for my oozie action. > I have 100's of jars in Oozie shared lib, including joda-time-1.6 jar. > In my oozie action (Pig action) I need joda 2.6 version, but the > classpath has both the jars. Is there any better way to load all 70+ > jars from existing share dlib, but override/replace one jar. > > Regards > Prashanth > > > > > -- _____________________________________________________________ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. ________________________________ NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
