It looks like there's a classpath issue (notice the HiveUtils error there).
Most likely because you have two versions
of es-hadoop in your classpath (2.1.0.Beta1 and 1.3.0.M1).
Use only one - I suggest 2.1.0.Beta1.
Cheers,
On 9/5/14 3:39 PM, Mohit Kumar Yadav wrote:
hi folks,
I facing following error while load data into elasticsearch using hive query.
ERROR:-
14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG method=Driver.execute
from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 INFO ql.Driver: Starting command: CREATE EXTERNAL TABLE
eslogs (time STRING, extension STRING,
clientip STRING, request STRING, response INT, agent STRING)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' =
'http://192.168.186.1','es.port' = '9200')
14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service address:
http://sandbox.hortonworks.com:8188/ws/v1/timeline/
14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook
14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG
method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook
from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG
method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook
start=1409389504583 end=1409389504583 duration=0
from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG method=TimeToSubmit
start=1409389502353 end=1409389504583 duration=2230
from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG method=runTasks
from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG method=task.DDL.Stage-0
from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 INFO exec.DDLTask: Use StorageHandler-supplied
org.elasticsearch.hadoop.hive.EsSerDe for table eslogs
14/08/30 02:05:04 ERROR exec.DDLTask: java.lang.IllegalAccessError:
org/elasticsearch/hadoop/hive/HiveUtils
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:80)
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:339)
at
org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:283)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:276)
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:626)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:593)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1504)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1271)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:349)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service address:
http://sandbox.hortonworks.com:8188/ws/v1/timeline/
14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook
14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG
method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook
from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG
method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook
start=1409389504698 end=1409389504698 duration=0
from=org.apache.hadoop.hive.ql.Driver>
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. org/elasticsearch/hadoop/hive/HiveUtils
14/08/30 02:05:04 ERROR ql.Driver: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
org/elasticsearch/hadoop/hive/HiveUtils
14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG method=Driver.execute
start=1409389504462 end=1409389504700
duration=238 from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 ERROR beeswax.BeeswaxServiceImpl: Exception while processing
query
BeeswaxException(message:Driver returned: 1. Errors: OK
converting to local
hdfs://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar
<http://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar>
Added
/tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841_resources/elasticsearch-hadoop-2.1.0.Beta1.jar
to class path
Added resource:
/tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841_resources/elasticsearch-hadoop-2.1.0.Beta1.jar
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. org/elasticsearch/hadoop/hive/HiveUtils
, log_context:d0960413-bfb9-4bdf-b615-26ad7821587b,
handle:QueryHandle(id:d0960413-bfb9-4bdf-b615-26ad7821587b,
log_context:d0960413-bfb9-4bdf-b615-26ad7821587b), SQLState: )
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:356)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
My setup:-
1. Hortonworks Sandbox VM (running on vmware player):- Hortonworks_Sandbox_2.1
2. elasticsearch up & running :- Elasticsearch version 1.3.1
3. elasticsearch-hadoop jars :-
elasticsearch-hadoop-2.1.0.Beta1,elasticsearch-hadoop-1.3.0.M1
I am following this url :-
https://github.com/hortonworks/hadoop-tutorials/blob/master/Community/T07_Elasticsearch_Hadoop_Integration.md
Hive Queries:-
Create table in hive and load data from zip file:-
CREATE TABLE apachelog (type STRING, time STRING, ext STRING, ip STRING, req
STRING, res INT, bytes INT, phpmem INT,
agent STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';
LOAD DATA INPATH '/user/hue/apache/apache.log' OVERWRITE INTO TABLE apachelog;
Load data into easlticsearch query:-
CREATE EXTERNAL TABLE eslogs (time STRING, extension STRING, clientip STRING,
request STRING, response INT, agent STRING)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' =
'http://192.168.186.1','es.port' = '9200');
INSERT OVERWRITE TABLE eslogs SELECT s.time, s.ext, s.ip, s.req, s.res, s.agent
FROM apachelog s;
Kindly suggest me where I am doing wrong and what is the meaning of this error
and why it is coming?
Regrads
Mohit Kumar Yadav
(MCA/BBA)
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
[email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Costin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5409AFE5.9010904%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
