Hi again,
And the issue I have is :
2014-08-06 22:06:37,083 FATAL [IPC Server handler 0 on 47378]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1407355085865_0004_m_000000_0 - exited :
java.lang.ClassNotFoundException: routines.StringHandling
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at mapreduce.MyMapper.map(MyMapper.java:14)
at mapreduce.MyMapper.map(MyMapper.java:1)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapreduce.lib.chain.Chain.runMapper(Chain.java:389)
at
org.apache.hadoop.mapreduce.lib.chain.ChainMapper.run(ChainMapper.java:149)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Remy Dubois
Component team manager and Big Data principal
Tel +33 1 46 25 06 00 | Mobile +33 6 11 73 56 72 | Email
rdub...@talend.com<mailto:rdub...@talend.com> | Skype remy.dubois.talend
Talend SA | 9, rue Pagès - 92150 Suresnes - France | +33 1 46 25 06 00 |
www.talend.com<http://www.talend.com>
Global Leader in Open Source Integration
[http://www.talend.com/sites/default/files/logo-talend.jpg]<http://www.talend.com/>
De : Remy Dubois [mailto:rdub...@talend.com]
Envoyé : mercredi 6 août 2014 20:19
À : user@hive.apache.org
Objet : RE: WebHCat and MapReduce JAR
Hi,
Thanks a lot for your answer. Following some information about what I do.
WebHCat version: Hive 0.13 embedded in HDP 2.1 GA.
The webhcat logs don’t show anything interesting.
REST call:
[cid:image002.jpg@01CFB1B4.8C0E22C0]
Jar location on HDFS:
[hdfs@hdp21 root]$ hadoop fs -ls /tmp/systemRoutines.jar
-rw-r--r-- 3 rdubois rdubois 264008 2014-08-06 19:16
/tmp/systemRoutines.jar
[hdfs@hdp21 root]$ hadoop fs -get /tmp/systemRoutines.jar /home/hdfs
[hdfs@hdp21 root]$ unzip -l /home/hdfs/systemRoutines.jar
Archive: /home/hdfs/systemRoutines.jar
Length Date Time Name
-------- ---- ---- ----
102 08-06-14 14:39 META-INF/MANIFEST.MF
3609 08-06-14 14:39 routines/StringHandling.class
Thanks,
Remy Dubois
Component team manager and Big Data principal
Tel +33 1 46 25 06 00 | Mobile +33 6 11 73 56 72 | Email
rdub...@talend.com<mailto:rdub...@talend.com> | Skype remy.dubois.talend
Talend SA | 9, rue Pagès - 92150 Suresnes - France | +33 1 46 25 06 00 |
www.talend.com<http://www.talend.com>
Global Leader in Open Source Integration
[http://www.talend.com/sites/default/files/logo-talend.jpg]<http://www.talend.com/>
De : Eugene Koifman [mailto:ekoif...@hortonworks.com]
Envoyé : mercredi 6 août 2014 19:43
À : user@hive.apache.org<mailto:user@hive.apache.org>
Objet : Re: WebHCat and MapReduce JAR
libjars as form param should work. Is there anything relevant in webhcat.log?
What version of webhcat are you using? What is the exact REST call you are
making?
On Wed, Aug 6, 2014 at 10:17 AM, Remy Dubois
<rdub...@talend.com<mailto:rdub...@talend.com>> wrote:
Hi,
I’m trying to submit a MapReduce job through webhcat using the
templeton/v1/mapreduce/jar path.
My job depends on third party libraries. I have tried a lot of things such as
using the libjars FormParam (the jar is located on HDFS), but it doesn’t work,
I got ClassNotFoundException.
I’ve workaround my issue merging the job jar and the dependencies jars, but
doing that is not really possible with my use case.
Can you please let me know if what I’m trying to do is possible ? I am able to
make the same job work, using the ToolRunner API (with the –libjars parameter)
without any problem.
Thanks a lot in advance,
Remy Dubois
Component team manager and Big Data principal
Tel +33 1 46 25 06 00 | Mobile +33 6 11 73 56 72 | Email
rdub...@talend.com<mailto:rdub...@talend.com> | Skype remy.dubois.talend
Talend SA | 9, rue Pagès - 92150 Suresnes - France | +33 1 46 25 06
00<tel:%2B33%201%2046%2025%2006%2000> | www.talend.com<http://www.talend.com>
Global Leader in Open Source Integration
[http://www.talend.com/sites/default/files/logo-talend.jpg]<http://www.talend.com/>
--
Thanks,
Eugene
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader of
this message is not the intended recipient, you are hereby notified that any
printing, copying, dissemination, distribution, disclosure or forwarding of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete it
from your system. Thank You.
