I tried to run the command curl -X GET http://engine-server:9200
and I'm getting a JSON response but when I'm running the command pio train I got org.elasticsearch.hadoop.rest.EsHadoopTransportException: java.net.NoRouteToHostException: No route to host (Host unreachable) exception 2017-04-07 9:18 GMT+04:00 Marius Rabenarivo <[email protected]>: > I tried using these configurations in the sparkConf section of engine.json > It has the same effect as the code I mentioned below but I still getting > the same exception. > > 2017-04-07 5:03 GMT+04:00 Pat Ferrel <[email protected]>: > >> BTW the docs for sparkConf are here: http://actionml.com/docs >> /ur_config#ur-spark-conf >> >> >> On Apr 6, 2017, at 4:52 PM, Pat Ferrel <[email protected]> wrote: >> >> PIO in the past used the “transport client” of Elasticsearch, which is on >> port 9300 by default, The UR uses the REST client for some things like >> rapid writes to ES using Spark using a lib supplied by Elasticsearch. >> >> So, Spark needs to have config that the Elasticsearch library can read. >> This can be done most easily by putting it into a sparkConf section of >> engine.json >> >> Like this: >> >> "sparkConf": { >> "spark.serializer": "org.apache.spark.serializer.KryoSerializer", >> "spark.kryo.registrator": "org.apache.mahout.sparkbindings.io >> .MahoutKryoRegistrator", >> "spark.kryo.referenceTracking": "false", >> "spark.kryoserializer.buffer": "300m", >> "es.index.auto.create": “true”, >> “es.nodes”: “node1,node2” <========== add this >> }, >> >> The list is comma separated host ip addresses of DNS names so no spaces. >> If you are using the default port 9200 you don’t need to specify it. These >> values are defined by Elasticsearch so see their docs if you have any >> question about what can be configured in this way. For instance you can >> also specify password and username in this manner. >> >> >> On Apr 6, 2017, at 4:27 PM, Marius Rabenarivo <[email protected]> >> wrote: >> >> Hello, >> >> I try to run training from a remote machine. >> >> It fails in later stage due to this exception : >> >> org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection >> error (check network and/or proxy settings)- all nodes failed; tried >> [[engine-server:9300]] >> >> According to the stack trace the error is coming from this line : >> >> https://github.com/actionml/universal-recommender/blob/maste >> r/src/main/scala/EsClient.scala#L179 >> >> It was using a default configuration to access ES. >> >> I tried to replace this line with : >> >> >> val elasticSearchConf = Storage.getConfig("ELASTICSEARCH").get >> >> val conf = Map( >> "es.mapping.id" -> "id", >> "es.nodes" -> elasticSearchConf.properties.get("HOSTS").head, >> "es.port" -> elasticSearchConf.properties.get("PORTS").head >> ) >> >> indexRDD.saveToEs(newIndexURI, conf) >> >> but I still getting the same error. >> >> Regards, >> >> Marius >> >> -- >> You received this message because you are subscribed to the Google Groups >> "actionml-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6NaV10Eh1 >> S4_t1Ub4PcA%40mail.gmail.com >> <https://groups.google.com/d/msgid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6NaV10Eh1S4_t1Ub4PcA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "actionml-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667%40occ >> amsmachete.com >> <https://groups.google.com/d/msgid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667%40occamsmachete.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> >> >
