We use a clustername different from hostname all the time. In fact if you have more than one node you will have to make them different since there are multiple hostnames and one clustername.
The clustername should be something unique to your application because ES has the sometimes helpful, but often maddeningly bad and hidden, habit of merging nodes into a cluster by clustername. For instance the default for ES is to merge all nodes on a LAN into one cluster. Anyway can you share your engine.json and the result of curl with exactly the same hostname? You need to add the clustername to pio-env.sh too so can you share that as well? On Apr 7, 2017, at 12:25 AM, Marius Rabenarivo <[email protected]> wrote: It works when using the default port 9200 and by changing the Elasticsearch cluster name to the same as the hostname as suggested here http://stackoverflow.com/questions/25885411/elasticsearch-error-failed-to-connect-to-master-no-route-to-host <http://stackoverflow.com/questions/25885411/elasticsearch-error-failed-to-connect-to-master-no-route-to-host> 2017-04-07 9:57 GMT+04:00 Marius Rabenarivo <[email protected] <mailto:[email protected]>>: I tried to run the command curl -X GET http://engine-server:9200 <http://engine-server:9200/> and I'm getting a JSON response but when I'm running the command pio train I got org.elasticsearch.hadoop.rest.EsHadoopTransportException: java.net <http://java.net/>.NoRouteToHostException: No route to host (Host unreachable) exception 2017-04-07 9:18 GMT+04:00 Marius Rabenarivo <[email protected] <mailto:[email protected]>>: I tried using these configurations in the sparkConf section of engine.json It has the same effect as the code I mentioned below but I still getting the same exception. 2017-04-07 5:03 GMT+04:00 Pat Ferrel <[email protected] <mailto:[email protected]>>: BTW the docs for sparkConf are here: http://actionml.com/docs/ur_config#ur-spark-conf <http://actionml.com/docs/ur_config#ur-spark-conf> On Apr 6, 2017, at 4:52 PM, Pat Ferrel <[email protected] <mailto:[email protected]>> wrote: PIO in the past used the “transport client” of Elasticsearch, which is on port 9300 by default, The UR uses the REST client for some things like rapid writes to ES using Spark using a lib supplied by Elasticsearch. So, Spark needs to have config that the Elasticsearch library can read. This can be done most easily by putting it into a sparkConf section of engine.json Like this: "sparkConf": { "spark.serializer": "org.apache.spark.serializer.KryoSerializer", "spark.kryo.registrator": "org.apache.mahout.sparkbindings.io <http://sparkbindings.io/>.MahoutKryoRegistrator", "spark.kryo.referenceTracking": "false", "spark.kryoserializer.buffer": "300m", "es.index.auto.create": “true”, “es.nodes”: “node1,node2” <========== add this }, The list is comma separated host ip addresses of DNS names so no spaces. If you are using the default port 9200 you don’t need to specify it. These values are defined by Elasticsearch so see their docs if you have any question about what can be configured in this way. For instance you can also specify password and username in this manner. On Apr 6, 2017, at 4:27 PM, Marius Rabenarivo <[email protected] <mailto:[email protected]>> wrote: Hello, I try to run training from a remote machine. It fails in later stage due to this exception : org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[engine-server:9300]] According to the stack trace the error is coming from this line : https://github.com/actionml/universal-recommender/blob/master/src/main/scala/EsClient.scala#L179 <https://github.com/actionml/universal-recommender/blob/master/src/main/scala/EsClient.scala#L179> It was using a default configuration to access ES. I tried to replace this line with : val elasticSearchConf = Storage.getConfig("ELASTICSEARCH").get val conf = Map( "es.mapping.id <http://es.mapping.id/>" -> "id", "es.nodes" -> elasticSearchConf.properties.get("HOSTS").head, "es.port" -> elasticSearchConf.properties.get("PORTS").head ) indexRDD.saveToEs(newIndexURI, conf) but I still getting the same error. Regards, Marius -- You received this message because you are subscribed to the Google Groups "actionml-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected] <mailto:[email protected]>. To post to this group, send email to [email protected] <mailto:[email protected]>. To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6NaV10Eh1S4_t1Ub4PcA%40mail.gmail.com <https://groups.google.com/d/msgid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6NaV10Eh1S4_t1Ub4PcA%40mail.gmail.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "actionml-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected] <mailto:[email protected]>. To post to this group, send email to [email protected] <mailto:[email protected]>. To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667%40occamsmachete.com <https://groups.google.com/d/msgid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667%40occamsmachete.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "actionml-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected] <mailto:[email protected]>. To post to this group, send email to [email protected] <mailto:[email protected]>. To view this discussion on the web visit https://groups.google.com/d/msgid/actionml-user/CAC-ATVGHNYHJ-V78B1knbpqWGn_cpkvYCC_%2BW3Tmdw6kywcSTw%40mail.gmail.com <https://groups.google.com/d/msgid/actionml-user/CAC-ATVGHNYHJ-V78B1knbpqWGn_cpkvYCC_%2BW3Tmdw6kywcSTw%40mail.gmail.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
