I tried using these configurations in the sparkConf section of engine.json It has the same effect as the code I mentioned below but I still getting the same exception.
2017-04-07 5:03 GMT+04:00 Pat Ferrel <[email protected]>: > BTW the docs for sparkConf are here: http://actionml.com/ > docs/ur_config#ur-spark-conf > > > On Apr 6, 2017, at 4:52 PM, Pat Ferrel <[email protected]> wrote: > > PIO in the past used the “transport client” of Elasticsearch, which is on > port 9300 by default, The UR uses the REST client for some things like > rapid writes to ES using Spark using a lib supplied by Elasticsearch. > > So, Spark needs to have config that the Elasticsearch library can read. > This can be done most easily by putting it into a sparkConf section of > engine.json > > Like this: > > "sparkConf": { > "spark.serializer": "org.apache.spark.serializer.KryoSerializer", > "spark.kryo.registrator": "org.apache.mahout.sparkbindings.io. > MahoutKryoRegistrator", > "spark.kryo.referenceTracking": "false", > "spark.kryoserializer.buffer": "300m", > "es.index.auto.create": “true”, > “es.nodes”: “node1,node2” <========== add this > }, > > The list is comma separated host ip addresses of DNS names so no spaces. > If you are using the default port 9200 you don’t need to specify it. These > values are defined by Elasticsearch so see their docs if you have any > question about what can be configured in this way. For instance you can > also specify password and username in this manner. > > > On Apr 6, 2017, at 4:27 PM, Marius Rabenarivo <[email protected]> > wrote: > > Hello, > > I try to run training from a remote machine. > > It fails in later stage due to this exception : > > org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection > error (check network and/or proxy settings)- all nodes failed; tried > [[engine-server:9300]] > > According to the stack trace the error is coming from this line : > > https://github.com/actionml/universal-recommender/blob/ > master/src/main/scala/EsClient.scala#L179 > > It was using a default configuration to access ES. > > I tried to replace this line with : > > > val elasticSearchConf = Storage.getConfig("ELASTICSEARCH").get > > val conf = Map( > "es.mapping.id" -> "id", > "es.nodes" -> elasticSearchConf.properties.get("HOSTS").head, > "es.port" -> elasticSearchConf.properties.get("PORTS").head > ) > > indexRDD.saveToEs(newIndexURI, conf) > > but I still getting the same error. > > Regards, > > Marius > > -- > You received this message because you are subscribed to the Google Groups > "actionml-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6NaV10Eh1S4_t1Ub4PcA% > 40mail.gmail.com > <https://groups.google.com/d/msgid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6NaV10Eh1S4_t1Ub4PcA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "actionml-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667% > 40occamsmachete.com > <https://groups.google.com/d/msgid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667%40occamsmachete.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > >
