Hello Donald and Team, I am working POC and I would like to use predictionIO.I know its configuration issue with elasticsearch but I am kind of stuck with below error so reaching out for help.
I am in need of some quick hand here as the time is running out. anything you feel I can try out or steps would be helpful please. *2018-03-07 21:36:15,602 ERROR org.apache.predictionio.tools.console.Console$ [main] - None of the configured nodes are available: [] (org.elasticsearch.client.transport.NoNodeAvailableException)* *org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: []* *I am using the steps from - http://predictionio.apache.org/install/install-sourcecode/ <http://predictionio.apache.org/install/install-sourcecode/>* *below is pio-env.sh and Error logs* *[mapr@valvcshad004vm conf]$ cat pio-env.sh* #!/usr/bin/env bash # # Copy this file as pio-env.sh and edit it for your site's configuration. # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # PredictionIO Main Configuration # # This section controls core behavior of PredictionIO. It is very likely that # you need to change these to fit your site. # SPARK_HOME: Apache Spark is a hard dependency and must be configured. #SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6 SPARK_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6 POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.2.1.jar MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.37.jar # ES_CONF_DIR: You must configure this if you have advanced configuration for # your Elasticsearch setup. # ES_CONF_DIR=/opt/elasticsearch # HADOOP_CONF_DIR: You must configure this if you intend to run PredictionIO # with Hadoop 2. # HADOOP_CONF_DIR=/opt/hadoop # HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO # with HBase on a remote cluster. # HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf # Filesystem paths where PredictionIO uses as block storage. PIO_FS_BASEDIR=$HOME/.pio_store PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp # PredictionIO Storage Configuration # # This section controls programs that make use of PredictionIO's built-in # storage facilities. Default values are shown below. # # For more information on storage configuration please refer to # http://predictionio.incubator.apache.org/system/anotherdatastore/ # Storage Repositories # Default is to use PostgreSQL #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL PIO_STORAGE_REPOSITORIES_METADATA_NAME=predictionio_metadata PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=predictionio_eventdata PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_ PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS # Storage Data Sources # PostgreSQL Default Settings # Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio # MySQL Example # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio # Elasticsearch Example PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch #PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300 PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2 # Local File System Example PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models # HBase Example PIO_STORAGE_SOURCES_HBASE_TYPE=hbase PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/hbase-1.2.6 *ERROR in the logs---* 2018-03-07 21:36:14,491 INFO org.apache.predictionio.data.storage.Storage$ [main] - Verifying Meta Data Backend (Source: ELASTICSEARCH)... 2018-03-07 21:36:15,601 ERROR org.apache.predictionio.tools.console.Console$ [main] - Unable to connect to all storage backends successfully. The following shows the error message from the storage backend. 2018-03-07 21:36:15,602 ERROR org.apache.predictionio.tools.console.Console$ [main] - None of the configured nodes are available: [] (org.elasticsearch.client.transport.NoNodeAvailableException) org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [] at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:305) at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:200) at org.elasticsearch.client.transport.support.InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86) at org.elasticsearch.client.support.AbstractIndicesAdminClient.exists(AbstractIndicesAdminClient.java:178) at org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:53) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91) at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65) at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:73) at org.apache.predictionio.data.storage.elasticsearch.ESEngineInstances.<init>(ESEngineInstances.scala:42) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:306) at org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:266) at org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:367) at org.apache.predictionio.data.storage.Storage$.verifyAllDataObjects(Storage.scala:342) at org.apache.predictionio.tools.console.Console$.status(Console.scala:1087) at org.apache.predictionio.tools.console.Console$$anonfun$main$1.apply(Console.scala:737) at org.apache.predictionio.tools.console.Console$$anonfun$main$1.apply(Console.scala:696) at scala.Option.map(Option.scala:145) at org.apache.predictionio.tools.console.Console$.main(Console.scala:696) at org.apache.predictionio.tools.console.Console.main(Console.scala) 2018-03-07 21:36:15,605 ERROR org.apache.predictionio.tools.console.Console$ [main] - Dumping configuration of initialized storage backend sources. Please make sure they are correct. 2018-03-07 21:36:15,607 ERROR org.apache.predictionio.tools.console.Console$ [main] - Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOSTS -> localhost, TYPE -> elasticsearch, HOME -> /dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2, PORTS -> 9300 2018-03-07 21:36:42,649 INFO org.apache.predictionio.tools.console.Console$ [main] - Creating Event Server at 0.0.0.0:7070 2018-03-07 21:36:43,417 ERROR org.apache.predictionio.data.storage.Storage$ [main] - Error initializing storage client for source HBASE java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration at org.apache.predictionio.data.storage.hbase.StorageClient.<init>(StorageClient.scala:46) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.predictionio.data.storage.Storage$.getClient(Storage.scala:220) at org.apache.predictionio.data.storage.Storage$.org$apache$predictionio$data$storage$Storage$$updateS2CM(Storage.scala:251) at org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:212) at org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:212) On Wed, Mar 7, 2018 at 1:21 AM, Pawan Agnihotri <pawan.agniho...@gmail.com> wrote: > Hello, > > I need your help to configure predictionIO on linux 7.2 - > > I am using > http://predictionio.apache.org/install/install-sourcecode/ link for > steps and installed spark, elastic search and hbase but getting below error > > > [mapr@valvcshad004vm bin]$ ./pio status > /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89: > /opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such file > or directory > /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90: > generate_compatible_classpath: command not found > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in [jar:file:/dfs/pawan_scala/ > mapr-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT. > jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in [jar:file:/data/opt/mapr/lib/ > slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > [INFO] [Console$] Inspecting PredictionIO... > [INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at > /dfs/pawan_scala/mapr-predictionio > [INFO] [Console$] Inspecting Apache Spark... > [INFO] [Console$] Apache Spark is installed at /dfs/pawan_scala/mapr- > predictionio/vendors/spark-2.1.1-bin-hadoop2.6 > [INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum requirement > of 1.3.0) > [INFO] [Console$] Inspecting storage backend connections... > *[WARN] [Storage$] There is no properly configured repository.* > *[ERROR] [Storage$] Required repository (METADATA) configuration is > missing.* > *[ERROR] [Storage$] There were 1 configuration errors. Exiting.* > [mapr@valvcshad004vm bin]$ > > Here is my *pio-env.sh *file > > [mapr@valvcshad004vm conf]$ cat pio-env.sh > # Default is to use PostgreSQL > #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta > #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL > > #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event > #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL > > #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model > #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL > > # Storage Data Sources > > # PostgreSQL Default Settings > # Please change "pio" to your database name in > PIO_STORAGE_SOURCES_PGSQL_URL > # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and > # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly > #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc > #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio > #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio > #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio > > # MySQL Example > # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc > # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio > # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio > # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio > > # Elasticsearch Example > PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch > PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch_cluster_name > PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost > PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300 > PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_ > scala/mapr-predictionio/vendors/elasticsearch-5.5.2 > > # Local File System Example > PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs > PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models > > # HBase Example > PIO_STORAGE_SOURCES_HBASE_TYPE=hbase > PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr- > predictionio/vendors/hbase-1.2.6 > > [mapr@valvcshad004vm conf]$ > > > -- > Thanks, > Pawan Agnihotri > -- Thanks, Pawan Agnihotri
