> > Ended Job = job_201008311250_0006 with errors Check your hadoop task logs, you will find more detailed information there.
-Shirjeet On Wed, Sep 1, 2010 at 6:13 AM, Robert Hennig <[email protected]> wrote: > Hello, > > I'm relative new to hive & hadoop and I have written a custom InputFormat > to be able to read our logfiles. I think I got everything right but when I > try to execute a query on a Amazon EMR cluster it fails with some error > messages that don't tell me what exactly is wrong. > > So this is the query I execute: > > add jar s3://amg.hadoop/hiveLib/hive-json-serde-0.1.jar; > add jar s3://amg.hadoop/hiveLib/hadoop-jar-with-dependencies.jar; > > DROP TABLE event_log; > > CREATE EXTERNAL TABLE IF NOT EXISTS event_log ( > EVENT_SUBTYPE STRING, > EVENT_TYPE STRING > ) > ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde' > STORED AS > INPUTFORMAT 'com.adconion.hadoop.hive.DataLogInputFormat' > OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' > LOCATION 's3://amg-events/2010/07/01/01'; > > SELECT event_type FROM event_log WHERE event_type = 'pp' LIMIT 10; > > Which results in the following output: > > had...@domu-12-31-39-0f-45-b3:~$ hive -f test.ql > Hive history > file=/mnt/var/lib/hive/tmp/history/hive_job_log_hadoop_201009011303_427866099.txt > Testing s3://amg.hadoop/hiveLib/hive-json-serde-0.1.jar > converting to local s3://amg.hadoop/hiveLib/hive-json-serde-0.1.jar > Added > /mnt/var/lib/hive/downloaded_resources/s3_amg.hadoop_hiveLib_hive-json-serde-0.1.jar > to class path > Testing s3://amg.hadoop/hiveLib/hadoop-jar-with-dependencies.jar > converting to local > s3://amg.hadoop/hiveLib/hadoop-jar-with-dependencies.jar > Added > /mnt/var/lib/hive/downloaded_resources/s3_amg.hadoop_hiveLib_hadoop-jar-with-dependencies.jar > to class path > OK > Time taken: 2.426 seconds > Found class for org.apache.hadoop.hive.contrib.serde2.JsonSerde > OK > Time taken: 0.332 seconds > Total MapReduce jobs = 1 > Launching Job 1 out of 1 > Number of reduce tasks is set to 0 since there's no reduce operator > Starting Job = job_201008311250_0006, Tracking URL = > http://domU-12-31-39-0F-45-B3.compute-1.internal:9100/jobdetails.jsp?jobid=job_201008311250_0006 > Kill Command = /home/hadoop/.versions/0.20/bin/../bin/hadoop job > -Dmapred.job.tracker=domU-12-31-39-0F-45-B3.compute-1.internal:9001 -kill > job_201008311250_0006 > 2010-09-01 13:04:04,376 Stage-1 map = 0%, reduce = 0% > 2010-09-01 13:04:34,681 Stage-1 map = 100%, reduce = 100% > Ended Job = job_201008311250_0006 with errors > > Failed tasks with most(4) failures : > Task URL: > http://domU-12-31-39-0F-45-B3.compute-1.internal:9100/taskdetails.jsp?jobid=job_201008311250_0006&tipid=task_201008311250_0006_m_000013 > > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.ExecDriver > > Only errors I can find under /mnt/var/log/apps/hive.log are multiple like > that one: > > 2010-09-01 13:03:36,586 DEBUG org.apache.hadoop.conf.Configuration > (Configuration.java:<init>(216)) - java.io.IOException: config() > at > org.apache.hadoop.conf.Configuration.<init>(Configuration.java:216) > at > org.apache.hadoop.conf.Configuration.<init>(Configuration.java:203) > at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:316) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:232) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > > And those errors: > > 2010-09-01 13:03:40,228 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.core.resources" but it cannot be resolved. > 2010-09-01 13:03:40,228 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.core.resources" but it cannot be resolved. > 2010-09-01 13:03:40,229 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.core.runtime" but it cannot be resolved. > 2010-09-01 13:03:40,229 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.core.runtime" but it cannot be resolved. > 2010-09-01 13:03:40,229 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.text" but it cannot be resolved. > 2010-09-01 13:03:40,229 ERROR DataNucleus.Plugin > (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires > "org.eclipse.text" but it cannot be resolved. > > Does anyone have an Idea what wents wrong? > > Thanks! > > Robert >
