I'm teaching myself hive and I'm having an issue with a simple select
statement.
I'm using Amazon's Elastic MapReduce.
I'm creating a table with some sample web log data I downloaded from a
sample given by Amazon.
I create this table:
CREATE EXTERNAL TABLE log_parser_stage (
remoteAddr STRING,
remoteLogname STRING,
user STRING,
time STRING,
request STRING,
status STRING,
bytes_string STRING,
referrer STRING,
browser STRING
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
"input.regex" = '^(\\S+) (\\S+) (\\S+)
\\[([\\w:/]+\\s[+\\-]\\d{4})\\] "(.+?)" (\\S+) (\\S+) "([^"]*)" "([^"]*)"',
"output.format.string" = "%1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s")
LOCATION 's3://mys3bucket/';
I then run a simple select * from the table to insure there is data there:
hive> SELECT * FROM log_parser_stage LIMIT 10;
OK
66.249.67.3 - - 20/Jul/2009:20:12:22 -0700 GET
/gallery/main.php?g2_controller=exif.SwitchDetailMode&g2_mode=detailed&g2_return=%2Fgallery%2Fmain.php%3Fg2_itemId%3D15741&g2_returnName=photo
HTTP/1.1 302 5 - Mozilla/5.0 (compatible;
Googlebot/2.1; +http://www.google.com/bot.html)
66.249.67.3 - - 20/Jul/2009:20:12:25 -0700 GET
/gallery/main.php?g2_itemId=15741&g2_fromNavId=x8fa12efc HTTP/1.1
200 8068 - Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)
64.233.172.17 - - 20/Jul/2009:20:12:26 -0700 GET
/gwidgets/alexa.xml HTTP/1.1 200 2969 - Mozilla/5.0
(compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)
74.125.74.193 - - 20/Jul/2009:20:13:01 -0700 GET
/gwidgets/alexa.xml HTTP/1.1 200 2969 - Mozilla/5.0
(compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)
192.168.1.198 - - 20/Jul/2009:20:13:18 -0700 GET /
HTTP/1.1 200 17935 - Mozilla/5.0 (Macintosh; U; Intel Mac
OS X 10_5_7; en-us) AppleWebKit/530.17 (KHTML, like Gecko) Version/4.0
Safari/530.17
192.168.1.198 - - 20/Jul/2009:20:13:18 -0700 GET
/style.css HTTP/1.1 200 1504 http://example.org/ Mozilla/5.0
(Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/530.17 (KHTML,
like Gecko) Version/4.0 Safari/530.17
192.168.1.198 - - 20/Jul/2009:20:13:19 -0700 GET
/favicon.ico HTTP/1.1 404 146 http://example.org/
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-us)
AppleWebKit/530.17 (KHTML, like Gecko) Version/4.0 Safari/530.17
66.249.67.3 - - 20/Jul/2009:20:13:21 -0700 GET
/gallery/main.php?g2_controller=exif.SwitchDetailMode&g2_mode=detailed&g2_return=%2Fgallery%2Fmain.php%3Fg2_itemId%3D30893&g2_returnName=photo
HTTP/1.1 302 5 - Mozilla/5.0 (compatible;
Googlebot/2.1; +http://www.google.com/bot.html)
66.249.67.3 - - 20/Jul/2009:20:13:24 -0700 GET
/gallery/main.php?g2_itemId=30893&g2_fromNavId=xfc647d65 HTTP/1.1
200 8196 - Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)
66.249.67.3 - - 20/Jul/2009:20:13:29 -0700 GET
/gallery/main.php?g2_view=search.SearchScan&g2_form%5BuseDefaultSettings%5D=1&g2_return=%2Fgallery%2Fmain.php%3Fg2_itemId%3D15789&g2_returnName=photo
HTTP/1.1 200 6360 - Mozilla/5.0 (compatible;
Googlebot/2.1; +http://www.google.com/bot.html)
Time taken: 13.954 seconds
I then try to query for two fields:
hive> SELECT remoteAddr, browser FROM log_parser_stage LIMIT 10;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201107062035_0007, Tracking URL =
http://ip-10-36-79-98.ec2.internal:9100/jobdetails.jsp?jobid=job_201107062035_0007
Kill Command = /home/hadoop/.versions/0.20/bin/../bin/hadoop job
-Dmapred.job.tracker=ip-10-36-79-98.ec2.internal:9001 -kill
job_201107062035_0007
2011-07-06 22:06:21,137 Stage-1 map = 0%, reduce = 0%
2011-07-06 22:07:05,974 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201107062035_0007 with errors
Failed tasks with most(4) failures :
Task URL:
http://ip-10-36-79-98.ec2.internal:9100/taskdetails.jsp?jobid=job_201107062035_0007&tipid=task_201107062035_0007_m_000000
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.ExecDriver
The following is from the job logs:
Meta VERSION="1" .
Job JOBID="job_201107062035_0006" JOBNAME="SELECT remoteAddr, browse\.\.\.log_parser_stage(Stage-1)"
USER="hadoop" SUBMIT_TIME="1309989443667"
JOBCONF="hdfs://ip-10-36-79-98\.ec2\.internal:9000/mnt/var/lib/hadoop/tmp/mapred/system/job_201107062035_0006/job\.xml"
.
Job JOBID="job_201107062035_0006" JOB_PRIORITY="NORMAL" .
Job JOBID="job_201107062035_0006" LAUNCH_TIME="1309989444146" TOTAL_MAPS="1"
TOTAL_REDUCES="0" JOB_STATUS="PREP" .
Task TASKID="task_201107062035_0006_m_000002" TASK_TYPE="SETUP"
START_TIME="1309989444665" SPLITS="" .
MapAttempt TASK_TYPE="SETUP" TASKID="task_201107062035_0006_m_000002"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000002_0" START_TIME="1309989445566"
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327" HTTP_PORT="9103" .
MapAttempt TASK_TYPE="SETUP" TASKID="task_201107062035_0006_m_000002" TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000002_0"
TASK_STATUS="SUCCESS" FINISH_TIME="1309989447674" HOSTNAME="/default-rack/ip-10-36-79-98\.ec2\.internal"
STATE_STRING="setup" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
Task TASKID="task_201107062035_0006_m_000002" TASK_TYPE="SETUP" TASK_STATUS="SUCCESS"
FINISH_TIME="1309989450674" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce
Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
Job JOBID="job_201107062035_0006" JOB_STATUS="RUNNING" .
Task TASKID="task_201107062035_0006_m_000000" TASK_TYPE="MAP" START_TIME="1309989450675"
SPLITS="/default-rack/localhost" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_0" START_TIME="1309989450701"
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327" HTTP_PORT="9103" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_0" TASK_STATUS="FAILED" FINISH_TIME="1309989457869"
HOSTNAME="ip-10-36-79-98\.ec2\.internal" ERROR="java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
\.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
\.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
\.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
at java\.security\.AccessController\.doPrivileged(Native Method)
at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
at java\.lang\.Class\.forName0(Native Method)
at java\.lang\.Class\.forName(Class\.java:247)
at
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
\.\.\. 19 more
" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_1" START_TIME="1309989459732"
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327" HTTP_PORT="9103" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_1" TASK_STATUS="FAILED" FINISH_TIME="1309989467109"
HOSTNAME="ip-10-36-79-98\.ec2\.internal" ERROR="java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
\.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
\.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
\.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
at java\.security\.AccessController\.doPrivileged(Native Method)
at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
at java\.lang\.Class\.forName0(Native Method)
at java\.lang\.Class\.forName(Class\.java:247)
at
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
\.\.\. 19 more
" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_2" START_TIME="1309989468821"
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327" HTTP_PORT="9103" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_2" TASK_STATUS="FAILED" FINISH_TIME="1309989477378"
HOSTNAME="ip-10-36-79-98\.ec2\.internal" ERROR="java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
\.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
\.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
\.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
at java\.security\.AccessController\.doPrivileged(Native Method)
at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
at java\.lang\.Class\.forName0(Native Method)
at java\.lang\.Class\.forName(Class\.java:247)
at
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
\.\.\. 19 more
" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_3" START_TIME="1309989477905"
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327" HTTP_PORT="9103" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_3" TASK_STATUS="FAILED" FINISH_TIME="1309989484902"
HOSTNAME="ip-10-36-79-98\.ec2\.internal" ERROR="java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
\.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
\.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
\.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
at java\.security\.AccessController\.doPrivileged(Native Method)
at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
at java\.lang\.Class\.forName0(Native Method)
at java\.lang\.Class\.forName(Class\.java:247)
at
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
\.\.\. 19 more
" .
Task TASKID="task_201107062035_0006_m_000000" TASK_TYPE="MAP" TASK_STATUS="FAILED"
FINISH_TIME="1309989484902" ERROR="java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
\.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
at
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
at
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
at
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
\.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
\.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
at
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
\.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException:
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
at java\.security\.AccessController\.doPrivileged(Native Method)
at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
at java\.lang\.Class\.forName0(Native Method)
at java\.lang\.Class\.forName(Class\.java:247)
at
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
at
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
\.\.\. 19 more
" TASK_ATTEMPT_ID="" .
Task TASKID="task_201107062035_0006_m_000001" TASK_TYPE="CLEANUP"
START_TIME="1309989486882" SPLITS="" .
MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201107062035_0006_m_000001"
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000001_0" START_TIME="1309989486898"
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327" HTTP_PORT="9103" .
MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201107062035_0006_m_000001" TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000001_0"
TASK_STATUS="SUCCESS" FINISH_TIME="1309989489057" HOSTNAME="/default-rack/ip-10-36-79-98\.ec2\.internal"
STATE_STRING="cleanup" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(SPILLED_RECORDS)(Spilled
Records)(0)]}" .
Task TASKID="task_201107062035_0006_m_000001" TASK_TYPE="CLEANUP" TASK_STATUS="SUCCESS"
FINISH_TIME="1309989489906" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce
Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
Job JOBID="job_201107062035_0006" FINISH_TIME="1309989489907" JOB_STATUS="FAILED"
FINISHED_MAPS="0" FINISHED_REDUCES="0" .
Any help is appreciated.
-Sal
