to add it globally to have better persistence and control, you could
add it in hive-site.xml
<property>
<name>hive.aux.jars.path</name>
<value>file:///${HIVE_HOME/auxlib/all_new.jar</value>
</property>
On Thu, Jul 7, 2011 at 2:09 PM, Sal Scalisi <sal...@hotmail.com> wrote:
>
> That did it, thanks!
>
> On 7/7/2011 1:30 PM, Norbert Burger wrote:
>
> There is likely a cleaner solution to this, but what happens if you add the
> Hive contrib JAR to the classpath at the beginning of your Hive sesssion?
>
> ADD JAR << fully-qualified path to HIVE contrib JAR >> , for example:
>
> ADD JAR /usr/lib/hive-0.7.0-cdh3u0/lib/hive-contrib-0.7.0-cdh3u0.jar
>
> Norbert
>
> On Thu, Jul 7, 2011 at 4:04 PM, Sal Scalisi <sal...@hotmail.com> wrote:
>>
>> I have some more information that may shed some light on this issue. I
>> re-ran my query with hive configured to log to the console window. I
>> noticed the following error from AWS:
>>
>> 11/07/07 19:56:42 INFO amazonaws.request: Sending Request: HEAD
>> https://mapreduce-dev.s3.amazonaws.com /log_parser%2Finput_%24folder%24
>> Headers: (Authorization: AWS XXXX=, Date: Thu, 07 Jul 2011 19:56:42 GMT,
>> Content-Type: application/x-www-form-urlencoded; charset=utf-8, )
>> 11/07/07 19:56:42 INFO amazonaws.request: Received error response: Status
>> Code: 404, AWS Request ID: XXXX, AWS Error Code: null, AWS Error Message:
>> Not Found, S3 Extended Request ID:
>>
>> I've tried to make my S3 folders public, but that did not resolve the
>> issue. If anyone has any insight, it is appreciated.
>>
>> -Sal
>>
>> On 7/6/2011 3:12 PM, Sal Scalisi wrote:
>> I'm teaching myself hive and I'm having an issue with a simple select
>> statement.
>>
>> I'm using Amazon's Elastic MapReduce.
>>
>> I'm creating a table with some sample web log data I downloaded from a
>> sample given by Amazon.
>>
>> I create this table:
>> CREATE EXTERNAL TABLE log_parser_stage (
>> remoteAddr STRING,
>> remoteLogname STRING,
>> user STRING,
>> time STRING,
>> request STRING,
>> status STRING,
>> bytes_string STRING,
>> referrer STRING,
>> browser STRING
>> )
>> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
>> WITH SERDEPROPERTIES (
>> "input.regex" = '^(\\S+) (\\S+) (\\S+) \\[([\\w:/]+\\s[+\\-]\\d{4})\\]
>> "(.+?)" (\\S+) (\\S+) "([^"]*)" "([^"]*)"',
>> "output.format.string" = "%1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s")
>> LOCATION 's3://mys3bucket/';
>>
>> I then run a simple select * from the table to insure there is data there:
>>
>> hive> SELECT * FROM log_parser_stage LIMIT 10;
>> OK
>> 66.249.67.3 - - 20/Jul/2009:20:12:22 -0700 GET
>> /gallery/main.php?g2_controller=exif.SwitchDetailMode&g2_mode=detailed&g2_return=%2Fgallery%2Fmain.php%3Fg2_itemId%3D15741&g2_returnName=photo
>> HTTP/1.1 302 5 - Mozilla/5.0 (compatible; Googlebot/2.1;
>> +http://www.google.com/bot.html)
>> 66.249.67.3 - - 20/Jul/2009:20:12:25 -0700 GET
>> /gallery/main.php?g2_itemId=15741&g2_fromNavId=x8fa12efc HTTP/1.1 200
>> 8068 - Mozilla/5.0 (compatible; Googlebot/2.1;
>> +http://www.google.com/bot.html)
>> 64.233.172.17 - - 20/Jul/2009:20:12:26 -0700 GET
>> /gwidgets/alexa.xml HTTP/1.1 200 2969 - Mozilla/5.0
>> (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)
>> 74.125.74.193 - - 20/Jul/2009:20:13:01 -0700 GET
>> /gwidgets/alexa.xml HTTP/1.1 200 2969 - Mozilla/5.0
>> (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)
>> 192.168.1.198 - - 20/Jul/2009:20:13:18 -0700 GET /
>> HTTP/1.1 200 17935 - Mozilla/5.0 (Macintosh; U; Intel Mac OS X
>> 10_5_7; en-us) AppleWebKit/530.17 (KHTML, like Gecko) Version/4.0
>> Safari/530.17
>> 192.168.1.198 - - 20/Jul/2009:20:13:18 -0700 GET
>> /style.css HTTP/1.1 200 1504 http://example.org/ Mozilla/5.0
>> (Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/530.17 (KHTML, like
>> Gecko) Version/4.0 Safari/530.17
>> 192.168.1.198 - - 20/Jul/2009:20:13:19 -0700 GET
>> /favicon.ico HTTP/1.1 404 146 http://example.org/
>> Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/530.17
>> (KHTML, like Gecko) Version/4.0 Safari/530.17
>> 66.249.67.3 - - 20/Jul/2009:20:13:21 -0700 GET
>> /gallery/main.php?g2_controller=exif.SwitchDetailMode&g2_mode=detailed&g2_return=%2Fgallery%2Fmain.php%3Fg2_itemId%3D30893&g2_returnName=photo
>> HTTP/1.1 302 5 - Mozilla/5.0 (compatible; Googlebot/2.1;
>> +http://www.google.com/bot.html)
>> 66.249.67.3 - - 20/Jul/2009:20:13:24 -0700 GET
>> /gallery/main.php?g2_itemId=30893&g2_fromNavId=xfc647d65 HTTP/1.1 200
>> 8196 - Mozilla/5.0 (compatible; Googlebot/2.1;
>> +http://www.google.com/bot.html)
>> 66.249.67.3 - - 20/Jul/2009:20:13:29 -0700 GET
>> /gallery/main.php?g2_view=search.SearchScan&g2_form%5BuseDefaultSettings%5D=1&g2_return=%2Fgallery%2Fmain.php%3Fg2_itemId%3D15789&g2_returnName=photo
>> HTTP/1.1 200 6360 - Mozilla/5.0 (compatible; Googlebot/2.1;
>> +http://www.google.com/bot.html)
>> Time taken: 13.954 seconds
>>
>> I then try to query for two fields:
>>
>> hive> SELECT remoteAddr, browser FROM log_parser_stage LIMIT 10;
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks is set to 0 since there's no reduce operator
>> Starting Job = job_201107062035_0007, Tracking URL =
>> http://ip-10-36-79-98.ec2.internal:9100/jobdetails.jsp?jobid=job_201107062035_0007
>> Kill Command = /home/hadoop/.versions/0.20/bin/../bin/hadoop job
>> -Dmapred.job.tracker=ip-10-36-79-98.ec2.internal:9001 -kill
>> job_201107062035_0007
>> 2011-07-06 22:06:21,137 Stage-1 map = 0%, reduce = 0%
>> 2011-07-06 22:07:05,974 Stage-1 map = 100%, reduce = 100%
>> Ended Job = job_201107062035_0007 with errors
>>
>> Failed tasks with most(4) failures :
>> Task URL:
>> http://ip-10-36-79-98.ec2.internal:9100/taskdetails.jsp?jobid=job_201107062035_0007&tipid=task_201107062035_0007_m_000000
>>
>> FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.ExecDriver
>>
>> The following is from the job logs:
>>
>> Meta VERSION="1" .
>> Job JOBID="job_201107062035_0006" JOBNAME="SELECT remoteAddr,
>> browse\.\.\.log_parser_stage(Stage-1)" USER="hadoop"
>> SUBMIT_TIME="1309989443667"
>> JOBCONF="hdfs://ip-10-36-79-98\.ec2\.internal:9000/mnt/var/lib/hadoop/tmp/mapred/system/job_201107062035_0006/job\.xml"
>> .
>> Job JOBID="job_201107062035_0006" JOB_PRIORITY="NORMAL" .
>> Job JOBID="job_201107062035_0006" LAUNCH_TIME="1309989444146" TOTAL_MAPS="1"
>> TOTAL_REDUCES="0" JOB_STATUS="PREP" .
>> Task TASKID="task_201107062035_0006_m_000002" TASK_TYPE="SETUP"
>> START_TIME="1309989444665" SPLITS="" .
>> MapAttempt TASK_TYPE="SETUP" TASKID="task_201107062035_0006_m_000002"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000002_0"
>> START_TIME="1309989445566"
>> TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"
>> HTTP_PORT="9103" .
>> MapAttempt TASK_TYPE="SETUP" TASKID="task_201107062035_0006_m_000002"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000002_0" TASK_STATUS="SUCCESS"
>> FINISH_TIME="1309989447674"
>> HOSTNAME="/default-rack/ip-10-36-79-98\.ec2\.internal" STATE_STRING="setup"
>> COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce
>> Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
>> Task TASKID="task_201107062035_0006_m_000002" TASK_TYPE="SETUP"
>> TASK_STATUS="SUCCESS" FINISH_TIME="1309989450674"
>> COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce
>> Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
>> Job JOBID="job_201107062035_0006" JOB_STATUS="RUNNING" .
>> Task TASKID="task_201107062035_0006_m_000000" TASK_TYPE="MAP"
>> START_TIME="1309989450675" SPLITS="/default-rack/localhost" .
>> MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_0"
>> START_TIME="1309989450701"
>> TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"
>> HTTP_PORT="9103" .
>> MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_0" TASK_STATUS="FAILED"
>> FINISH_TIME="1309989457869" HOSTNAME="ip-10-36-79-98\.ec2\.internal"
>> ERROR="java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
>> at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
>> at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 5 more
>> Caused by: java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
>> \.\.\. 10 more
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 13 more
>> Caused by: java\.lang\.RuntimeException: Map operator initialization failed
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
>> \.\.\. 18 more
>> Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
>> java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
>> \.\.\. 18 more
>> Caused by: java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
>> at java\.security\.AccessController\.doPrivileged(Native Method)
>> at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
>> at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
>> at java\.lang\.Class\.forName0(Native Method)
>> at java\.lang\.Class\.forName(Class\.java:247)
>> at
>> org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
>> \.\.\. 19 more
>> " .
>> MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_1"
>> START_TIME="1309989459732"
>> TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"
>> HTTP_PORT="9103" .
>> MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_1" TASK_STATUS="FAILED"
>> FINISH_TIME="1309989467109" HOSTNAME="ip-10-36-79-98\.ec2\.internal"
>> ERROR="java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
>> at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
>> at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 5 more
>> Caused by: java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
>> \.\.\. 10 more
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 13 more
>> Caused by: java\.lang\.RuntimeException: Map operator initialization failed
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
>> \.\.\. 18 more
>> Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
>> java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
>> \.\.\. 18 more
>> Caused by: java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
>> at java\.security\.AccessController\.doPrivileged(Native Method)
>> at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
>> at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
>> at java\.lang\.Class\.forName0(Native Method)
>> at java\.lang\.Class\.forName(Class\.java:247)
>> at
>> org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
>> \.\.\. 19 more
>> " .
>> MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_2"
>> START_TIME="1309989468821"
>> TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"
>> HTTP_PORT="9103" .
>> MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_2" TASK_STATUS="FAILED"
>> FINISH_TIME="1309989477378" HOSTNAME="ip-10-36-79-98\.ec2\.internal"
>> ERROR="java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
>> at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
>> at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 5 more
>> Caused by: java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
>> \.\.\. 10 more
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 13 more
>> Caused by: java\.lang\.RuntimeException: Map operator initialization failed
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
>> \.\.\. 18 more
>> Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
>> java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
>> \.\.\. 18 more
>> Caused by: java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
>> at java\.security\.AccessController\.doPrivileged(Native Method)
>> at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
>> at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
>> at java\.lang\.Class\.forName0(Native Method)
>> at java\.lang\.Class\.forName(Class\.java:247)
>> at
>> org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
>> \.\.\. 19 more
>> " .
>> MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_3"
>> START_TIME="1309989477905"
>> TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"
>> HTTP_PORT="9103" .
>> MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_3" TASK_STATUS="FAILED"
>> FINISH_TIME="1309989484902" HOSTNAME="ip-10-36-79-98\.ec2\.internal"
>> ERROR="java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
>> at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
>> at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 5 more
>> Caused by: java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
>> \.\.\. 10 more
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 13 more
>> Caused by: java\.lang\.RuntimeException: Map operator initialization failed
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
>> \.\.\. 18 more
>> Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
>> java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
>> \.\.\. 18 more
>> Caused by: java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
>> at java\.security\.AccessController\.doPrivileged(Native Method)
>> at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
>> at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
>> at java\.lang\.Class\.forName0(Native Method)
>> at java\.lang\.Class\.forName(Class\.java:247)
>> at
>> org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
>> \.\.\. 19 more
>> " .
>> Task TASKID="task_201107062035_0006_m_000000" TASK_TYPE="MAP"
>> TASK_STATUS="FAILED" FINISH_TIME="1309989484902"
>> ERROR="java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
>> at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
>> at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 5 more
>> Caused by: java\.lang\.RuntimeException: Error in configuring object
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
>> at org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
>> \.\.\. 10 more
>> Caused by: java\.lang\.reflect\.InvocationTargetException
>> at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
>> at
>> sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
>> at
>> sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
>> at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
>> at
>> org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
>> \.\.\. 13 more
>> Caused by: java\.lang\.RuntimeException: Map operator initialization failed
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
>> \.\.\. 18 more
>> Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException:
>> java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
>> \.\.\. 18 more
>> Caused by: java\.lang\.ClassNotFoundException:
>> org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
>> at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
>> at java\.security\.AccessController\.doPrivileged(Native Method)
>> at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
>> at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
>> at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
>> at java\.lang\.Class\.forName0(Native Method)
>> at java\.lang\.Class\.forName(Class\.java:247)
>> at
>> org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
>> at
>> org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
>> \.\.\. 19 more
>> " TASK_ATTEMPT_ID="" .
>> Task TASKID="task_201107062035_0006_m_000001" TASK_TYPE="CLEANUP"
>> START_TIME="1309989486882" SPLITS="" .
>> MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201107062035_0006_m_000001"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000001_0"
>> START_TIME="1309989486898"
>> TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"
>> HTTP_PORT="9103" .
>> MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201107062035_0006_m_000001"
>> TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000001_0" TASK_STATUS="SUCCESS"
>> FINISH_TIME="1309989489057"
>> HOSTNAME="/default-rack/ip-10-36-79-98\.ec2\.internal"
>> STATE_STRING="cleanup"
>> COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce
>> Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
>> Task TASKID="task_201107062035_0006_m_000001" TASK_TYPE="CLEANUP"
>> TASK_STATUS="SUCCESS" FINISH_TIME="1309989489906"
>> COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce
>> Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
>> Job JOBID="job_201107062035_0006" FINISH_TIME="1309989489907"
>> JOB_STATUS="FAILED" FINISHED_MAPS="0" FINISHED_REDUCES="0" .
>>
>> Any help is appreciated.
>>
>> -Sal
>
--
- Prash
