Yes, I set the clustername in Elasticsearch config too. Thank you for your advices about fields.
2017-04-10 19:07 GMT+04:00 Pat Ferrel <[email protected]>: > Do you have the clustername set in Elasticsearch config also? It is > changed in /usr/local/elasticsearch/config/elasticsearch.yml Setting this > in both places will decouple server name from cluster name. Unfortunately > changing a cluster name will call for a complete reset of PredictionIO data > since it will loose track of engine metadata. I have found it easiest to > export/backup then delete all apps and re-install or erase any table in > hbase and elasticsearch. You have to get the clustername right before you > start because it’s a pain to change. > > Also the fields must correspond to arrays of strings, not an array of true > or false, an array of one element, a string of “T” or “F” would work for > this. For fields the property must be attached to all items by using a $set > event, then your visible and sponsored will work more like what you expect. > They are not working at all now. > > > On Apr 9, 2017, at 10:43 PM, Marius Rabenarivo <[email protected]> > wrote: > > Hello, > > Find attached the pio-env.sh and engine.json > > that we are using. > > 2017-04-08 18:46 GMT+04:00 Marius Rabenarivo <[email protected]>: > >> I'll share them monday. >> >> 2017-04-07 22:23 GMT+04:00 Pat Ferrel <[email protected]>: >> >>> We use a clustername different from hostname all the time. In fact if >>> you have more than one node you will have to make them different since >>> there are multiple hostnames and one clustername. >>> >>> The clustername should be something unique to your application because >>> ES has the sometimes helpful, but often maddeningly bad and hidden, habit >>> of merging nodes into a cluster by clustername. For instance the default >>> for ES is to merge all nodes on a LAN into one cluster. >>> >>> Anyway can you share your engine.json and the result of curl with >>> exactly the same hostname? You need to add the clustername to pio-env.sh >>> too so can you share that as well? >>> >>> >>> On Apr 7, 2017, at 12:25 AM, Marius Rabenarivo < >>> [email protected]> wrote: >>> >>> It works when using the default port 9200 and by changing the >>> Elasticsearch cluster name to the same as the hostname as suggested here >>> >>> http://stackoverflow.com/questions/25885411/elasticsearch-er >>> ror-failed-to-connect-to-master-no-route-to-host >>> >>> 2017-04-07 9:57 GMT+04:00 Marius Rabenarivo <[email protected]> >>> : >>> >>>> I tried to run the command >>>> >>>> curl -X GET http://engine-server:9200 >>>> >>>> and I'm getting a JSON response but when I'm running the command >>>> >>>> pio train >>>> >>>> I got >>>> >>>> org.elasticsearch.hadoop.rest.EsHadoopTransportException: >>>> java.net.NoRouteToHostException: >>>> No route to host (Host unreachable) >>>> >>>> exception >>>> >>>> 2017-04-07 9:18 GMT+04:00 Marius Rabenarivo <[email protected] >>>> >: >>>> >>>>> I tried using these configurations in the sparkConf section of >>>>> engine.json >>>>> It has the same effect as the code I mentioned below but I still >>>>> getting the same exception. >>>>> >>>>> 2017-04-07 5:03 GMT+04:00 Pat Ferrel <[email protected]>: >>>>> >>>>>> BTW the docs for sparkConf are here: http://actionml.com/docs >>>>>> /ur_config#ur-spark-conf >>>>>> >>>>>> >>>>>> On Apr 6, 2017, at 4:52 PM, Pat Ferrel <[email protected]> wrote: >>>>>> >>>>>> PIO in the past used the “transport client” of Elasticsearch, which >>>>>> is on port 9300 by default, The UR uses the REST client for some things >>>>>> like rapid writes to ES using Spark using a lib supplied by >>>>>> Elasticsearch. >>>>>> >>>>>> So, Spark needs to have config that the Elasticsearch library can >>>>>> read. This can be done most easily by putting it into a sparkConf section >>>>>> of engine.json >>>>>> >>>>>> Like this: >>>>>> >>>>>> "sparkConf": { >>>>>> "spark.serializer": "org.apache.spark.serializer.KryoSerializer", >>>>>> "spark.kryo.registrator": "org.apache.mahout.sparkbindings.io >>>>>> .MahoutKryoRegistrator", >>>>>> "spark.kryo.referenceTracking": "false", >>>>>> "spark.kryoserializer.buffer": "300m", >>>>>> "es.index.auto.create": “true”, >>>>>> “es.nodes”: “node1,node2” <========== add this >>>>>> }, >>>>>> >>>>>> The list is comma separated host ip addresses of DNS names so no >>>>>> spaces. If you are using the default port 9200 you don’t need to specify >>>>>> it. These values are defined by Elasticsearch so see their docs if you >>>>>> have >>>>>> any question about what can be configured in this way. For instance you >>>>>> can >>>>>> also specify password and username in this manner. >>>>>> >>>>>> >>>>>> On Apr 6, 2017, at 4:27 PM, Marius Rabenarivo < >>>>>> [email protected]> wrote: >>>>>> >>>>>> Hello, >>>>>> >>>>>> I try to run training from a remote machine. >>>>>> >>>>>> It fails in later stage due to this exception : >>>>>> >>>>>> org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: >>>>>> Connection error (check network and/or proxy settings)- all nodes failed; >>>>>> tried [[engine-server:9300]] >>>>>> >>>>>> According to the stack trace the error is coming from this line : >>>>>> >>>>>> https://github.com/actionml/universal-recommender/blob/maste >>>>>> r/src/main/scala/EsClient.scala#L179 >>>>>> >>>>>> It was using a default configuration to access ES. >>>>>> >>>>>> I tried to replace this line with : >>>>>> >>>>>> >>>>>> val elasticSearchConf = Storage.getConfig("ELASTICSEARCH").get >>>>>> >>>>>> val conf = Map( >>>>>> "es.mapping.id" -> "id", >>>>>> "es.nodes" -> elasticSearchConf.properties.get("HOSTS").head, >>>>>> "es.port" -> elasticSearchConf.properties.get("PORTS").head >>>>>> ) >>>>>> >>>>>> indexRDD.saveToEs(newIndexURI, conf) >>>>>> >>>>>> but I still getting the same error. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Marius >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "actionml-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> To view this discussion on the web visit https://groups.google.co >>>>>> m/d/msgid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6Na >>>>>> V10Eh1S4_t1Ub4PcA%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6NaV10Eh1S4_t1Ub4PcA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "actionml-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> To view this discussion on the web visit https://groups.google.co >>>>>> m/d/msgid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667 >>>>>> %40occamsmachete.com >>>>>> <https://groups.google.com/d/msgid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667%40occamsmachete.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>>> >>>>> >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "actionml-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> To view this discussion on the web visit https://groups.google.co >>> m/d/msgid/actionml-user/CAC-ATVGHNYHJ-V78B1knbpqWGn_cpkvYCC_ >>> %2BW3Tmdw6kywcSTw%40mail.gmail.com >>> <https://groups.google.com/d/msgid/actionml-user/CAC-ATVGHNYHJ-V78B1knbpqWGn_cpkvYCC_%2BW3Tmdw6kywcSTw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >> > > -- > You received this message because you are subscribed to the Google Groups > "actionml-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/actionml-user/CAC-ATVG8madmx19TqijGra4n6TxDSnxvz > iB4RNggnvAosB7Etg%40mail.gmail.com > <https://groups.google.com/d/msgid/actionml-user/CAC-ATVG8madmx19TqijGra4n6TxDSnxvziB4RNggnvAosB7Etg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > <engine.json><pio-env.sh> > >
