Costin, About what you asked, 1) Hive server is running on the same machine as elasticsearch, 2) I get a response when I do a curl http://elasticsearc&hivehost:9200
I feel I'm missing something simple. This is what I've got until now. 1) First error (connection refused) is because i left the default settings as is. I changed the 'network.host' in elasticsearch.yml to the hostname 2) Second error (cannot discover elasticsearch version) is when I changed the 'network.host' in elasticsearch.yml to the hostname. Looked at the source code and figured I'm expected to pass eshost:port to the hive table at table creation. (https://github.com/elasticsearch/elasticsearch-hadoop/blob/master/src/main/java/org/elasticsearch/hadoop/hive/EsStorageHandler.java#L48) . So I included 'es.resource.write'='http://elasticsearchhost:9200' as one of the table properties during table creation step. STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES('es.resource' = 'esdemo/hive', 'es.mapping.name'='time:@timestamp', 'es.resource.write'='http://eshost:9200'); Now I get connection refused again. Is that the right way to pass that information ? Thanks for you patience and help On Friday, March 14, 2014 10:33:14 AM UTC-5, Costin Leau wrote: > > Without any extra information I'm afraid I can only guess what might be > the issue. > Make sure you have the latest Elasticsearch 0.90 or 1.x available on port > 9200 with the HTTP/REST port open. > Also make sure that Hive actually runs on the same machine - not just the > client but also the server (meaning Hadoop > itself). > > You indicate that if you change the network configuration you get an error > regarding the version - this suggests that: > > 1. Hive is actually running on a different machine than ES - hence the > network error > 2. After pointing Hive to the actual ES machine, you get an error since > you're using an old Elasticsearch version (0.20) > > Cheers, > > On 3/14/2014 12:19 AM, P lva wrote: > > I have a simple query > > insert into table eslogs select * from eslogs_ext; > > Hive and elasticsearch are running on the same host. > > > > To execute the script I'm following the directions from the link. > > > http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html > > > > There are two elasticsearch nodes, and they can recognize each other (as > indicated by start up process) , but why would > > hive not be able to pick them up ? Can you explain what could have gone > wrong ? > > > > > > On Thursday, March 13, 2014 4:28:14 PM UTC-5, Costin Leau wrote: > > > > What does your Hive script look like? Can you confirm the ip/address > of your Hive and Elasticsearch ? How are you > > executing the script? > > The error indicates an error in your network configuration. > > > > Cheers, > > > > P.S. Feel free to post a gist or whatever it's convenient. > > > > On 3/13/2014 10:38 PM, P lva wrote: > > > Hi, I have few weblogs in a hive table that I'd like to visualize > in kibana. > > > ES is on the same node as hive server. > > > > > > Followed directions from this pagehttp:// > www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html > > < > http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html> > > > > > > > I can create a table using esstorage handler, but when I tried to > ingest data into this table I got > > > > > > Error: java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing > > > row {***first row of my table**} > > > at > org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175) > > > at > org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > > > at > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429) > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > > > at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162) > > > at java.security.AccessController.doPrivileged(Native > Method) > > > at javax.security.auth.Subject.doAs(Subject.java:415) > > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) > > > > > at > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157) > > > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive > Runtime Error while processing row {*** first row of > > > my table**} > > > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > java.io.IOException: Out of nodes and retries; caught exception > > > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652) > > > > > at > org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504) > > > at > org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842) > > > at > org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88) > > > > > at > org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504) > > > at > org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842) > > > at > org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91) > > > > > at > org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504) > > > at > org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842) > > > at > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534) > > > ... 9 more > > > Caused by: java.io.IOException: Out of nodes and retries; caught > exception > > > at > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81) > > > at > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221) > > > at > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205) > > > at > org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209) > > > at > org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103) > > > at > org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85) > > > at > org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60) > > > > > at > org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165) > > > > > at > org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50) > > > > > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638) > > > > > ... 18 more > > > Caused by: java.net.ConnectException: Connection refused > > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > > at > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > > > > at > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > > > > > at > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > > > at > java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > > at java.net.Socket.connect(Socket.java:579) > > > at java.net.Socket.connect(Socket.java:528) > > > at java.net.Socket.<init>(Socket.java:425) > > > at java.net.Socket.<init>(Socket.java:280) > > > at > > > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80) > > > > > at > > > > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122) > > > > > at > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > > > at > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > > > > > at > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > > > > > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > > > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > > > at > org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160) > > > > > at > org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74) > > > ... 27 more > > > > > > Now, I changed the config network.host to the ipadress of the > server. Now when I run hive insert data i get > > > FAILED: IllegalStateException Cannot discover Elasticsearch > version > > > > > > > > > -- > > > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, > send an email to > > >[email protected] <javascript:> <mailto: > [email protected] <javascript:> <javascript:>>. > > > To view this discussion on the web visit > > > > https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com > > > < > https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com> > > > > > < > https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer > > > < > https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>>. > > > > > > > For more options, visithttps://groups.google.com/d/optout < > https://groups.google.com/d/optout>. > > > > -- > > Costin > > > > -- > > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to > > [email protected] <javascript:> <mailto: > [email protected] <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com > > > < > https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > > For more options, visit https://groups.google.com/d/optout. > > -- > Costin > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
