Re: Unable to connect to all storage backends successfully

[Jim Miller]

Hi Guys,

The template that I am using is*
incubator-predictionio-template-recommender.*

Thanks,

Jim

On Wed, Sep 27, 2017 at 3:49 PM, Jim Miller  wrote:

> incubator-predictionio-template-recommender
>
>
> On Wed, Sep 27, 2017 at 2:25 PM, Donald Szeto  wrote:
>
>> What template and version are you using?
>>
>> On Tue, Sep 26, 2017 at 9:28 PM Jim Miller 
>> wrote:
>>
>>> Hi guys,
>>>
>>> I have another error that I cannot get past:
>>>
>>> *vagrant:MyRecommendation/ (develop*) $ pio train
>>>
>>>
>>>  [21:47:39]*
>>> *[ERROR] [Console$] unable to read engine factory from
>>> /home/vagrant/MyRecommendation/engine.json. Aborting.*
>>>
>>>
>>> Does anyone have any advice?
>>>
>>> Thanks,
>>>
>>> Jim
>>>
>>> --
>>> Jim Miller
>>>
>>> On September 26, 2017 at 7:42:02 PM, Jim Miller (jemiller1...@gmail.com)
>>> wrote:
>>>
>>> Shinsuke, you rock!  That was it.
>>>
>>> Thank you so much,
>>>
>>> Jim
>>>
>>> --
>>> Jim Miller
>>>
>>> On September 24, 2017 at 7:29:28 PM, Shinsuke Sugaya (
>>> shinsuke.sug...@gmail.com) wrote:
>>>
>>> Hi,
>>>
>>> > PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>>>
>>> Elasticsearch 5 uses 9200 port.
>>> Could you try 9200 port?
>>>
>>> Thanks,
>>> shinsuke
>>>
>>>
>>> 2017-09-24 3:02 GMT+09:00 Jim Miller :
>>> > Hi Donald,
>>> >
>>> > Tried just now and received the following error:
>>> >
>>> > vagrant:~/ $ pio status
>>> > [13:34:52]
>>> > [INFO] [Management$] Inspecting PredictionIO...
>>> > [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at
>>> > /home/vagrant/pio/PredictionIO-0.12.0-incubating
>>> > [INFO] [Management$] Inspecting Apache Spark...
>>> > [INFO] [Management$] Apache Spark is installed at
>>> > /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/
>>> spark-2.1.1-bin-hadoop2.7
>>> > [INFO] [Management$] Apache Spark 2.1.1 detected (meets minimum
>>> requirement
>>> > of 1.3.0)
>>> > [INFO] [Management$] Inspecting storage backend connections...
>>> > [INFO] [Storage$] Verifying Meta Data Backend (Source:
>>> ELASTICSEARCH)...
>>> > [ERROR] [Management$] Unable to connect to all storage backends
>>> > successfully.
>>> > The following shows the error message from the storage backend.
>>> >
>>> > error while performing request (java.lang.RuntimeException)
>>> >
>>> > Dumping configuration of initialized storage backend sources.
>>> > Please make sure they are correct.
>>> >
>>> > Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME ->
>>> > /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/ela
>>> sticsearch-5.5.2,
>>> > HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstCluster, TYPE ->
>>> > elastic search
>>> >
>>> >
>>> > HERE IS MY PIO-ENV.SH
>>> >
>>> > # PredictionIO Main Configuration
>>> > #
>>> > # This section controls core behavior of PredictionIO. It is very
>>> likely
>>> > that
>>> > # you need to change these to fit your site.
>>> >
>>> > # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
>>> > # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
>>> > SPARK_HOME=$PIO_HOME/vendors/spark-2.1.1-bin-hadoop2.7
>>> >
>>> > POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
>>> > MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
>>> >
>>> > # ES_CONF_DIR: You must configure this if you have advanced
>>> configuration
>>> > for
>>> > # your Elasticsearch setup.
>>> > ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-5.5.2
>>> > # HADOOP_CONF_DIR=/opt/hadoop
>>> >
>>> > # HBASE_CONF_DIR: You must configure this if you intend to run
>>> PredictionIO
>>> > # with HBase on a remote cluster.
>>> > HBASE_CONF_DIR=$PIO_HOME/hbase-1.3.1/conf
>>> >
>>> > # Filesystem paths where PredictionIO uses as block storage.
>>> > PIO_FS_BASEDIR=$HOME/.pio_store
>>> > PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
>>> > PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>>> >
>>> > # PredictionIO Storage Configuration
>>> > #
>>> > # This section controls programs that make use of PredictionIO's
>>> built-in
>>> > # storage facilities. Default values are shown below.
>>> > #
>>> > # For more information on storage configuration please refer to
>>> > # http://predictionio.incubator.apache.org/system/anotherdatastore/
>>> >
>>> > # Storage Repositories
>>> >
>>> > # Default is to use PostgreSQL
>>> > PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>>> > PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>>> >
>>> > PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>>> > PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>>> >
>>> > PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>>> > PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
>>> >
>>> > # Storage Data Sources
>>> >
>>> > # PostgreSQL Default Settings
>>> > # Please change "pio" to your database name in
>>> PIO_STORAGE_SOURCES_PGSQL_URL
>>> > # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>>> > # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>>> > # 

Re: Unable to connect to all storage backends successfully

[Jim Miller]

incubator-predictionio-template-recommender


On Wed, Sep 27, 2017 at 2:25 PM, Donald Szeto  wrote:

> What template and version are you using?
>
> On Tue, Sep 26, 2017 at 9:28 PM Jim Miller  wrote:
>
>> Hi guys,
>>
>> I have another error that I cannot get past:
>>
>> *vagrant:MyRecommendation/ (develop*) $ pio train
>>
>>  [21:47:39]*
>> *[ERROR] [Console$] unable to read engine factory from
>> /home/vagrant/MyRecommendation/engine.json. Aborting.*
>>
>>
>> Does anyone have any advice?
>>
>> Thanks,
>>
>> Jim
>>
>> --
>> Jim Miller
>>
>> On September 26, 2017 at 7:42:02 PM, Jim Miller (jemiller1...@gmail.com)
>> wrote:
>>
>> Shinsuke, you rock!  That was it.
>>
>> Thank you so much,
>>
>> Jim
>>
>> --
>> Jim Miller
>>
>> On September 24, 2017 at 7:29:28 PM, Shinsuke Sugaya (
>> shinsuke.sug...@gmail.com) wrote:
>>
>> Hi,
>>
>> > PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>>
>> Elasticsearch 5 uses 9200 port.
>> Could you try 9200 port?
>>
>> Thanks,
>> shinsuke
>>
>>
>> 2017-09-24 3:02 GMT+09:00 Jim Miller :
>> > Hi Donald,
>> >
>> > Tried just now and received the following error:
>> >
>> > vagrant:~/ $ pio status
>> > [13:34:52]
>> > [INFO] [Management$] Inspecting PredictionIO...
>> > [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at
>> > /home/vagrant/pio/PredictionIO-0.12.0-incubating
>> > [INFO] [Management$] Inspecting Apache Spark...
>> > [INFO] [Management$] Apache Spark is installed at
>> > /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-2.1.
>> 1-bin-hadoop2.7
>> > [INFO] [Management$] Apache Spark 2.1.1 detected (meets minimum
>> requirement
>> > of 1.3.0)
>> > [INFO] [Management$] Inspecting storage backend connections...
>> > [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
>> > [ERROR] [Management$] Unable to connect to all storage backends
>> > successfully.
>> > The following shows the error message from the storage backend.
>> >
>> > error while performing request (java.lang.RuntimeException)
>> >
>> > Dumping configuration of initialized storage backend sources.
>> > Please make sure they are correct.
>> >
>> > Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME ->
>> > /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/
>> elasticsearch-5.5.2,
>> > HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstCluster, TYPE ->
>> > elastic search
>> >
>> >
>> > HERE IS MY PIO-ENV.SH
>> >
>> > # PredictionIO Main Configuration
>> > #
>> > # This section controls core behavior of PredictionIO. It is very likely
>> > that
>> > # you need to change these to fit your site.
>> >
>> > # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
>> > # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
>> > SPARK_HOME=$PIO_HOME/vendors/spark-2.1.1-bin-hadoop2.7
>> >
>> > POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
>> > MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
>> >
>> > # ES_CONF_DIR: You must configure this if you have advanced
>> configuration
>> > for
>> > # your Elasticsearch setup.
>> > ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-5.5.2
>> > # HADOOP_CONF_DIR=/opt/hadoop
>> >
>> > # HBASE_CONF_DIR: You must configure this if you intend to run
>> PredictionIO
>> > # with HBase on a remote cluster.
>> > HBASE_CONF_DIR=$PIO_HOME/hbase-1.3.1/conf
>> >
>> > # Filesystem paths where PredictionIO uses as block storage.
>> > PIO_FS_BASEDIR=$HOME/.pio_store
>> > PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
>> > PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>> >
>> > # PredictionIO Storage Configuration
>> > #
>> > # This section controls programs that make use of PredictionIO's
>> built-in
>> > # storage facilities. Default values are shown below.
>> > #
>> > # For more information on storage configuration please refer to
>> > # http://predictionio.incubator.apache.org/system/anotherdatastore/
>> >
>> > # Storage Repositories
>> >
>> > # Default is to use PostgreSQL
>> > PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>> > PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>> >
>> > PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>> > PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>> >
>> > PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>> > PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
>> >
>> > # Storage Data Sources
>> >
>> > # PostgreSQL Default Settings
>> > # Please change "pio" to your database name in
>> PIO_STORAGE_SOURCES_PGSQL_URL
>> > # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>> > # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>> > # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
>> > # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
>> > # PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
>> > # PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>> >
>> > # MySQL Example
>> > # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
>> > # 

Re: Unable to connect to all storage backends successfully

[Donald Szeto]

What template and version are you using?

On Tue, Sep 26, 2017 at 9:28 PM Jim Miller  wrote:

> Hi guys,
>
> I have another error that I cannot get past:
>
> *vagrant:MyRecommendation/ (develop*) $ pio train
>
>  [21:47:39]*
> *[ERROR] [Console$] unable to read engine factory from
> /home/vagrant/MyRecommendation/engine.json. Aborting.*
>
>
> Does anyone have any advice?
>
> Thanks,
>
> Jim
>
> --
> Jim Miller
>
> On September 26, 2017 at 7:42:02 PM, Jim Miller (jemiller1...@gmail.com)
> wrote:
>
> Shinsuke, you rock!  That was it.
>
> Thank you so much,
>
> Jim
>
> --
> Jim Miller
>
> On September 24, 2017 at 7:29:28 PM, Shinsuke Sugaya (
> shinsuke.sug...@gmail.com) wrote:
>
> Hi,
>
> > PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>
> Elasticsearch 5 uses 9200 port.
> Could you try 9200 port?
>
> Thanks,
> shinsuke
>
>
> 2017-09-24 3:02 GMT+09:00 Jim Miller :
> > Hi Donald,
> >
> > Tried just now and received the following error:
> >
> > vagrant:~/ $ pio status
> > [13:34:52]
> > [INFO] [Management$] Inspecting PredictionIO...
> > [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at
> > /home/vagrant/pio/PredictionIO-0.12.0-incubating
> > [INFO] [Management$] Inspecting Apache Spark...
> > [INFO] [Management$] Apache Spark is installed at
> >
> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-2.1.1-bin-hadoop2.7
> > [INFO] [Management$] Apache Spark 2.1.1 detected (meets minimum
> requirement
> > of 1.3.0)
> > [INFO] [Management$] Inspecting storage backend connections...
> > [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
> > [ERROR] [Management$] Unable to connect to all storage backends
> > successfully.
> > The following shows the error message from the storage backend.
> >
> > error while performing request (java.lang.RuntimeException)
> >
> > Dumping configuration of initialized storage backend sources.
> > Please make sure they are correct.
> >
> > Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME ->
> >
> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-5.5.2,
> > HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstCluster, TYPE ->
> > elastic search
> >
> >
> > HERE IS MY PIO-ENV.SH
> >
> > # PredictionIO Main Configuration
> > #
> > # This section controls core behavior of PredictionIO. It is very likely
> > that
> > # you need to change these to fit your site.
> >
> > # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
> > # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
> > SPARK_HOME=$PIO_HOME/vendors/spark-2.1.1-bin-hadoop2.7
> >
> > POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
> > MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
> >
> > # ES_CONF_DIR: You must configure this if you have advanced configuration
> > for
> > # your Elasticsearch setup.
> > ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-5.5.2
> > # HADOOP_CONF_DIR=/opt/hadoop
> >
> > # HBASE_CONF_DIR: You must configure this if you intend to run
> PredictionIO
> > # with HBase on a remote cluster.
> > HBASE_CONF_DIR=$PIO_HOME/hbase-1.3.1/conf
> >
> > # Filesystem paths where PredictionIO uses as block storage.
> > PIO_FS_BASEDIR=$HOME/.pio_store
> > PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
> > PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
> >
> > # PredictionIO Storage Configuration
> > #
> > # This section controls programs that make use of PredictionIO's built-in
> > # storage facilities. Default values are shown below.
> > #
> > # For more information on storage configuration please refer to
> > # http://predictionio.incubator.apache.org/system/anotherdatastore/
> >
> > # Storage Repositories
> >
> > # Default is to use PostgreSQL
> > PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
> > PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
> >
> > PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> > PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
> >
> > PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
> > PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
> >
> > # Storage Data Sources
> >
> > # PostgreSQL Default Settings
> > # Please change "pio" to your database name in
> PIO_STORAGE_SOURCES_PGSQL_URL
> > # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
> > # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
> > # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
> > # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
> > # PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
> > # PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
> >
> > # MySQL Example
> > # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
> > # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
> > # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
> > # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
> >
> > # Elasticsearch Example
> > # PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> > # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
> > # 

Re: Unable to connect to all storage backends successfully

[Jim Miller]

Hi guys,

I have another error that I cannot get past:

vagrant:MyRecommendation/ (develop*) $ pio train                                
                                                                                
                                                      [21:47:39]
[ERROR] [Console$] unable to read engine factory from 
/home/vagrant/MyRecommendation/engine.json. Aborting.

Does anyone have any advice?

Thanks,

Jim

-- 
Jim Miller

On September 26, 2017 at 7:42:02 PM, Jim Miller (jemiller1...@gmail.com) wrote:

Shinsuke, you rock!  That was it.

Thank you so much,

Jim

-- 
Jim Miller

On September 24, 2017 at 7:29:28 PM, Shinsuke Sugaya 
(shinsuke.sug...@gmail.com) wrote:

Hi,

> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300

Elasticsearch 5 uses 9200 port.
Could you try 9200 port?

Thanks,
shinsuke


2017-09-24 3:02 GMT+09:00 Jim Miller :
> Hi Donald,
>
> Tried just now and received the following error:
>
> vagrant:~/ $ pio status
> [13:34:52]
> [INFO] [Management$] Inspecting PredictionIO...
> [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at
> /home/vagrant/pio/PredictionIO-0.12.0-incubating
> [INFO] [Management$] Inspecting Apache Spark...
> [INFO] [Management$] Apache Spark is installed at
> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-2.1.1-bin-hadoop2.7
> [INFO] [Management$] Apache Spark 2.1.1 detected (meets minimum requirement
> of 1.3.0)
> [INFO] [Management$] Inspecting storage backend connections...
> [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
> [ERROR] [Management$] Unable to connect to all storage backends
> successfully.
> The following shows the error message from the storage backend.
>
> error while performing request (java.lang.RuntimeException)
>
> Dumping configuration of initialized storage backend sources.
> Please make sure they are correct.
>
> Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME ->
> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-5.5.2,
> HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstCluster, TYPE ->
> elastic search
>
>
> HERE IS MY PIO-ENV.SH
>
> # PredictionIO Main Configuration
> #
> # This section controls core behavior of PredictionIO. It is very likely
> that
> # you need to change these to fit your site.
>
> # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
> # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
> SPARK_HOME=$PIO_HOME/vendors/spark-2.1.1-bin-hadoop2.7
>
> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
>
> # ES_CONF_DIR: You must configure this if you have advanced configuration
> for
> # your Elasticsearch setup.
> ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-5.5.2
> # HADOOP_CONF_DIR=/opt/hadoop
>
> # HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO
> # with HBase on a remote cluster.
> HBASE_CONF_DIR=$PIO_HOME/hbase-1.3.1/conf
>
> # Filesystem paths where PredictionIO uses as block storage.
> PIO_FS_BASEDIR=$HOME/.pio_store
> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>
> # PredictionIO Storage Configuration
> #
> # This section controls programs that make use of PredictionIO's built-in
> # storage facilities. Default values are shown below.
> #
> # For more information on storage configuration please refer to
> # http://predictionio.incubator.apache.org/system/anotherdatastore/
>
> # Storage Repositories
>
> # Default is to use PostgreSQL
> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>
> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>
> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
>
> # Storage Data Sources
>
> # PostgreSQL Default Settings
> # Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL
> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
> # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
> # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
> # PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>
> # MySQL Example
> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>
> # Elasticsearch Example
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
> #
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2
> # Optional basic HTTP auth
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name
> # 

Re: Unable to connect to all storage backends successfully

[Jim Miller]

Shinsuke, you rock!  That was it.

Thank you so much,

Jim

-- 
Jim Miller

On September 24, 2017 at 7:29:28 PM, Shinsuke Sugaya 
(shinsuke.sug...@gmail.com) wrote:

Hi,  

> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300  

Elasticsearch 5 uses 9200 port.  
Could you try 9200 port?  

Thanks,  
shinsuke  


2017-09-24 3:02 GMT+09:00 Jim Miller :  
> Hi Donald,  
>  
> Tried just now and received the following error:  
>  
> vagrant:~/ $ pio status  
> [13:34:52]  
> [INFO] [Management$] Inspecting PredictionIO...  
> [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at  
> /home/vagrant/pio/PredictionIO-0.12.0-incubating  
> [INFO] [Management$] Inspecting Apache Spark...  
> [INFO] [Management$] Apache Spark is installed at  
> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-2.1.1-bin-hadoop2.7
>   
> [INFO] [Management$] Apache Spark 2.1.1 detected (meets minimum requirement  
> of 1.3.0)  
> [INFO] [Management$] Inspecting storage backend connections...  
> [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...  
> [ERROR] [Management$] Unable to connect to all storage backends  
> successfully.  
> The following shows the error message from the storage backend.  
>  
> error while performing request (java.lang.RuntimeException)  
>  
> Dumping configuration of initialized storage backend sources.  
> Please make sure they are correct.  
>  
> Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME ->  
> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-5.5.2, 
>  
> HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstCluster, TYPE ->  
> elastic search  
>  
>  
> HERE IS MY PIO-ENV.SH  
>  
> # PredictionIO Main Configuration  
> #  
> # This section controls core behavior of PredictionIO. It is very likely  
> that  
> # you need to change these to fit your site.  
>  
> # SPARK_HOME: Apache Spark is a hard dependency and must be configured.  
> # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7  
> SPARK_HOME=$PIO_HOME/vendors/spark-2.1.1-bin-hadoop2.7  
>  
> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar  
> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar  
>  
> # ES_CONF_DIR: You must configure this if you have advanced configuration  
> for  
> # your Elasticsearch setup.  
> ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-5.5.2  
> # HADOOP_CONF_DIR=/opt/hadoop  
>  
> # HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO  
> # with HBase on a remote cluster.  
> HBASE_CONF_DIR=$PIO_HOME/hbase-1.3.1/conf  
>  
> # Filesystem paths where PredictionIO uses as block storage.  
> PIO_FS_BASEDIR=$HOME/.pio_store  
> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines  
> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp  
>  
> # PredictionIO Storage Configuration  
> #  
> # This section controls programs that make use of PredictionIO's built-in  
> # storage facilities. Default values are shown below.  
> #  
> # For more information on storage configuration please refer to  
> # http://predictionio.incubator.apache.org/system/anotherdatastore/  
>  
> # Storage Repositories  
>  
> # Default is to use PostgreSQL  
> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta  
> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH  
>  
> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event  
> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE  
>  
> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model  
> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS  
>  
> # Storage Data Sources  
>  
> # PostgreSQL Default Settings  
> # Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL  
> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and  
> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly  
> # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc  
> # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio  
> # PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio  
> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio  
>  
> # MySQL Example  
> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc  
> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio  
> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio  
> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio  
>  
> # Elasticsearch Example  
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch  
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost  
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200  
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http  
> #  
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2  
> # Optional basic HTTP auth  
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name  
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_PASSWORD=my-secret  
> # Elasticsearch 1.x Example  
> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch  
> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=firstCluster  
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost  
> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300  
> 

Re: Unable to connect to all storage backends successfully

[Shinsuke Sugaya]

Hi,

> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300

Elasticsearch 5 uses 9200 port.
Could you try 9200 port?

Thanks,
 shinsuke


2017-09-24 3:02 GMT+09:00 Jim Miller :
> Hi Donald,
>
> Tried just now and received the following error:
>
> vagrant:~/ $ pio status
> [13:34:52]
> [INFO] [Management$] Inspecting PredictionIO...
> [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at
> /home/vagrant/pio/PredictionIO-0.12.0-incubating
> [INFO] [Management$] Inspecting Apache Spark...
> [INFO] [Management$] Apache Spark is installed at
> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-2.1.1-bin-hadoop2.7
> [INFO] [Management$] Apache Spark 2.1.1 detected (meets minimum requirement
> of 1.3.0)
> [INFO] [Management$] Inspecting storage backend connections...
> [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
> [ERROR] [Management$] Unable to connect to all storage backends
> successfully.
> The following shows the error message from the storage backend.
>
> error while performing request (java.lang.RuntimeException)
>
> Dumping configuration of initialized storage backend sources.
> Please make sure they are correct.
>
> Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME ->
> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-5.5.2,
> HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstCluster, TYPE ->
> elastic search
>
>
> HERE IS MY PIO-ENV.SH
>
> # PredictionIO Main Configuration
> #
> # This section controls core behavior of PredictionIO. It is very likely
> that
> # you need to change these to fit your site.
>
> # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
> # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
> SPARK_HOME=$PIO_HOME/vendors/spark-2.1.1-bin-hadoop2.7
>
> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
>
> # ES_CONF_DIR: You must configure this if you have advanced configuration
> for
> #  your Elasticsearch setup.
> ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-5.5.2
> # HADOOP_CONF_DIR=/opt/hadoop
>
> # HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO
> # with HBase on a remote cluster.
> HBASE_CONF_DIR=$PIO_HOME/hbase-1.3.1/conf
>
> # Filesystem paths where PredictionIO uses as block storage.
> PIO_FS_BASEDIR=$HOME/.pio_store
> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>
> # PredictionIO Storage Configuration
> #
> # This section controls programs that make use of PredictionIO's built-in
> # storage facilities. Default values are shown below.
> #
> # For more information on storage configuration please refer to
> # http://predictionio.incubator.apache.org/system/anotherdatastore/
>
> # Storage Repositories
>
> # Default is to use PostgreSQL
> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>
> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>
> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
>
> # Storage Data Sources
>
> # PostgreSQL Default Settings
> # Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL
> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
> # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
> # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
> # PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>
> # MySQL Example
> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>
> # Elasticsearch Example
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
> #
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2
> # Optional basic HTTP auth
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name
> # PIO_STORAGE_SOURCES_ELASTICSEARCH_PASSWORD=my-secret
> # Elasticsearch 1.x Example
> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=firstCluster
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2
>
> # Local File System Example
> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>
> # HBase Example
> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
> PIO_STORAGE_SOURCES_HBASE_HOME=$PIO_HOME/vendors/hbase-1.3.1
>
> # AWS S3 Example
> # PIO_STORAGE_SOURCES_S3_TYPE=s3
> # 

Re: Unable to connect to all storage backends successfully

[Jim Miller]

Hi Donald,

Tried just now and received the following error:

vagrant:~/ $ pio status                                                         
                                             [13:34:52]
[INFO] [Management$] Inspecting PredictionIO...
[INFO] [Management$] PredictionIO 0.12.0-incubating is installed at 
/home/vagrant/pio/PredictionIO-0.12.0-incubating
[INFO] [Management$] Inspecting Apache Spark...
[INFO] [Management$] Apache Spark is installed at 
/home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-2.1.1-bin-hadoop2.7
[INFO] [Management$] Apache Spark 2.1.1 detected (meets minimum requirement of 
1.3.0)
[INFO] [Management$] Inspecting storage backend connections...
[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
[ERROR] [Management$] Unable to connect to all storage backends successfully.
The following shows the error message from the storage backend.

error while performing request (java.lang.RuntimeException)

Dumping configuration of initialized storage backend sources.
Please make sure they are correct.

Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> 
/home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-5.5.2, 
HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstCluster, TYPE -> elastic 
search


HERE IS MY PIO-ENV.SH
# PredictionIO Main Configuration
#
# This section controls core behavior of PredictionIO. It is very likely that
# you need to change these to fit your site.

# SPARK_HOME: Apache Spark is a hard dependency and must be configured.
# SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
SPARK_HOME=$PIO_HOME/vendors/spark-2.1.1-bin-hadoop2.7

POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar

# ES_CONF_DIR: You must configure this if you have advanced configuration for
#              your Elasticsearch setup.
ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-5.5.2
# HADOOP_CONF_DIR=/opt/hadoop

# HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO
#                 with HBase on a remote cluster.
HBASE_CONF_DIR=$PIO_HOME/hbase-1.3.1/conf

# Filesystem paths where PredictionIO uses as block storage.
PIO_FS_BASEDIR=$HOME/.pio_store
PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp

# PredictionIO Storage Configuration
#
# This section controls programs that make use of PredictionIO's built-in
# storage facilities. Default values are shown below.
#
# For more information on storage configuration please refer to
# http://predictionio.incubator.apache.org/system/anotherdatastore/

# Storage Repositories

# Default is to use PostgreSQL
PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH

PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE

PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS

# Storage Data Sources

# PostgreSQL Default Settings
# Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL
# Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
# PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
# PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
# PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio

# MySQL Example
# PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
# PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
# PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
# PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio

# Elasticsearch Example
# PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
# PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
# PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
# PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
# PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2
# Optional basic HTTP auth
# PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name
# PIO_STORAGE_SOURCES_ELASTICSEARCH_PASSWORD=my-secret
# Elasticsearch 1.x Example
PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=firstCluster
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2

# Local File System Example
PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models

# HBase Example
PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
PIO_STORAGE_SOURCES_HBASE_HOME=$PIO_HOME/vendors/hbase-1.3.1

# AWS S3 Example
# PIO_STORAGE_SOURCES_S3_TYPE=s3
# PIO_STORAGE_SOURCES_S3_BUCKET_NAME=pio_bucket
# PIO_STORAGE_SOURCES_S3_BASE_PATH=pio_model

ELASTICSEARCH.YML
# Elasticsearch Configuration =
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make 

Re: Unable to connect to all storage backends successfully

[Jim Miller]

Hi Donald,

I did not.  I will read the release notes and update accordingly.

Thanks!

Jim

-- 
Jim Miller

On September 20, 2017 at 1:01:53 PM, Donald Szeto (don...@apache.org) wrote:

Hey Jim,

Did you build PIO 0.12 with ES 1.4 support? ES 1.x is being deprecated in 0.12 
so the default build will use ES 5.x support.

See the upcoming release notes: 
https://github.com/apache/incubator-predictionio/blob/release/0.12.0/RELEASE.md#behavior-changes

Regards,
Donald

On Wed, Sep 20, 2017 at 8:14 AM, Jim Miller  wrote:
Yes.  I’m following this tutorial but using 0.12.0 instead of 0.10.0:
https://medium.freecodecamp.org/building-an-recommendation-engine-with-apache-prediction-io-ml-server-aed0319e0d8

-- 
Jim Miller

On September 20, 2017 at 10:51:39 AM, Pat Ferrel (p...@occamsmachete.com) wrote:

meaning is “firstcluster” the cluster name in your Elasticsearch configuration?


On Sep 19, 2017, at 8:54 PM, Vaghawan Ojha  wrote:

I think the problem is with Elasticsearch, are you sure the cluster exists in 
elasticsearch configuration? 

On Wed, Sep 20, 2017 at 8:17 AM, Jim Miller  wrote:
Hi,

I’m using PredictionIO 0.12.0-incubating with ElasticSearch and Hbase:
PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4
PredictionIO-0.12.0-incubating/vendors/hbase-1.0.0
PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6

All starts with no errors but with pio status I get:

[INFO] [Management$] Inspecting PredictionIO...
[INFO] [Management$] PredictionIO 0.12.0-incubating is installed at 
/home/vagrant/pio/PredictionIO-0.12.0-incubating
[INFO] [Management$] Inspecting Apache Spark...
[INFO] [Management$] Apache Spark is installed at 
/home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6
[INFO] [Management$] Apache Spark 1.5.1 detected (meets minimum requirement of 
1.3.0)
[INFO] [Management$] Inspecting storage backend connections...
[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
[ERROR] [Management$] Unable to connect to all storage backends successfully.
The following shows the error message from the storage backend.

Connection closed 
(org.apache.predictionio.shaded.org.apache.http.ConnectionClosedException)

Dumping configuration of initialized storage backend sources.
Please make sure they are correct.

Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> 
/home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4, 
HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstcluster, TYPE -> 
elasticsearch

Can anyone give me an idea of what I need to fix this issue?  Here is 

# PredictionIO Main Configuration
#
# This section controls core behavior of PredictionIO. It is very likely that
# you need to change these to fit your site.

# SPARK_HOME: Apache Spark is a hard dependency and must be configured.
# SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6

POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar

# ES_CONF_DIR: You must configure this if you have advanced configuration for
#              your Elasticsearch setup.
ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-1.4.4/conf

# HADOOP_CONF_DIR: You must configure this if you intend to run PredictionIO
#                  with Hadoop 2.
HADOOP_CONF_DIR=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6/conf

# HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO
#                 with HBase on a remote cluster.
HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf

# Filesystem paths where PredictionIO uses as block storage.
PIO_FS_BASEDIR=$HOME/.pio_store
PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp

# PredictionIO Storage Configuration
#
# This section controls programs that make use of PredictionIO's built-in
# storage facilities. Default values are shown below.
#
# For more information on storage configuration please refer to
# http://predictionio.incubator.apache.org/system/anotherdatastore/

# Storage Repositories

# Default is to use PostgreSQL
PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH

PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE

PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS

# Storage Data Sources

# PostgreSQL Default Settings
# Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL
# Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
# PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
# PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
# PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio

# MySQL Example
# 

Re: Unable to connect to all storage backends successfully

[Donald Szeto]

Hey Jim,

Did you build PIO 0.12 with ES 1.4 support? ES 1.x is being deprecated in
0.12 so the default build will use ES 5.x support.

See the upcoming release notes:
https://github.com/apache/incubator-predictionio/blob/release/0.12.0/RELEASE.md#behavior-changes

Regards,
Donald

On Wed, Sep 20, 2017 at 8:14 AM, Jim Miller  wrote:

> Yes.  I’m following this tutorial but using 0.12.0 instead of 0.10.0:
> https://medium.freecodecamp.org/building-an-recommendation-engine-with-
> apache-prediction-io-ml-server-aed0319e0d8
>
> --
> Jim Miller
>
> On September 20, 2017 at 10:51:39 AM, Pat Ferrel (p...@occamsmachete.com)
> wrote:
>
> meaning is “firstcluster” the cluster name in your Elasticsearch
> configuration?
>
>
> On Sep 19, 2017, at 8:54 PM, Vaghawan Ojha  wrote:
>
> I think the problem is with Elasticsearch, are you sure the cluster exists
> in elasticsearch configuration?
>
> On Wed, Sep 20, 2017 at 8:17 AM, Jim Miller 
> wrote:
>
>> Hi,
>>
>> I’m using PredictionIO 0.12.0-incubating with ElasticSearch and Hbase:
>>
>> PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4
>> PredictionIO-0.12.0-incubating/vendors/hbase-1.0.0
>> PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6
>>
>>
>> All starts with no errors but with pio status I get:
>>
>> [INFO] [Management$] Inspecting PredictionIO...
>> [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at
>> /home/vagrant/pio/PredictionIO-0.12.0-incubating
>> [INFO] [Management$] Inspecting Apache Spark...
>> [INFO] [Management$] Apache Spark is installed at
>> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/
>> spark-1.5.1-bin-hadoop2.6
>> [INFO] [Management$] Apache Spark 1.5.1 detected (meets minimum
>> requirement of 1.3.0)
>> [INFO] [Management$] Inspecting storage backend connections...
>> [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
>> [ERROR] [Management$] Unable to connect to all storage backends
>> successfully.
>> The following shows the error message from the storage backend.
>>
>> Connection closed (org.apache.predictionio.shade
>> d.org.apache.http.ConnectionClosedException)
>>
>> Dumping configuration of initialized storage backend sources.
>> Please make sure they are correct.
>>
>> Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME ->
>> /home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4,
>> HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstcluster, TYPE ->
>> elasticsearch
>>
>>
>> Can anyone give me an idea of what I need to fix this issue?  Here is
>>
>>
>> # PredictionIO Main Configuration
>> #
>> # This section controls core behavior of PredictionIO. It is very likely
>> that
>> # you need to change these to fit your site.
>>
>> # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
>> # SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
>> SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6
>>
>> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
>> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
>>
>> # ES_CONF_DIR: You must configure this if you have advanced configuration
>> for
>> #  your Elasticsearch setup.
>> ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-1.4.4/conf
>>
>> # HADOOP_CONF_DIR: You must configure this if you intend to run
>> PredictionIO
>> #  with Hadoop 2.
>> HADOOP_CONF_DIR=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6/conf
>>
>> # HBASE_CONF_DIR: You must configure this if you intend to run
>> PredictionIO
>> # with HBase on a remote cluster.
>> HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf
>>
>> # Filesystem paths where PredictionIO uses as block storage.
>> PIO_FS_BASEDIR=$HOME/.pio_store
>> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
>> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>>
>> # PredictionIO Storage Configuration
>> #
>> # This section controls programs that make use of PredictionIO's built-in
>> # storage facilities. Default values are shown below.
>> #
>> # For more information on storage configuration please refer to
>> # http://predictionio.incubator.apache.org/system/anotherdatastore/
>>
>> # Storage Repositories
>>
>> # Default is to use PostgreSQL
>> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>>
>> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>>
>> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
>>
>> # Storage Data Sources
>>
>> # PostgreSQL Default Settings
>> # Please change "pio" to your database name in
>> PIO_STORAGE_SOURCES_PGSQL_URL
>> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>> # PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
>> # PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
>> # 

Re: Unable to connect to all storage backends successfully

[Jim Miller]

Yes.  I’m following this tutorial but using 0.12.0 instead of 0.10.0:
https://medium.freecodecamp.org/building-an-recommendation-engine-with-apache-prediction-io-ml-server-aed0319e0d8

-- 
Jim Miller

On September 20, 2017 at 10:51:39 AM, Pat Ferrel (p...@occamsmachete.com) wrote:

meaning is “firstcluster” the cluster name in your Elasticsearch configuration?


On Sep 19, 2017, at 8:54 PM, Vaghawan Ojha  wrote:

I think the problem is with Elasticsearch, are you sure the cluster exists in 
elasticsearch configuration? 

On Wed, Sep 20, 2017 at 8:17 AM, Jim Miller  wrote:
Hi,

I’m using PredictionIO 0.12.0-incubating with ElasticSearch and Hbase:
PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4
PredictionIO-0.12.0-incubating/vendors/hbase-1.0.0
PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6

All starts with no errors but with pio status I get:

[INFO] [Management$] Inspecting PredictionIO...
[INFO] [Management$] PredictionIO 0.12.0-incubating is installed at 
/home/vagrant/pio/PredictionIO-0.12.0-incubating
[INFO] [Management$] Inspecting Apache Spark...
[INFO] [Management$] Apache Spark is installed at 
/home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6
[INFO] [Management$] Apache Spark 1.5.1 detected (meets minimum requirement of 
1.3.0)
[INFO] [Management$] Inspecting storage backend connections...
[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
[ERROR] [Management$] Unable to connect to all storage backends successfully.
The following shows the error message from the storage backend.

Connection closed 
(org.apache.predictionio.shaded.org.apache.http.ConnectionClosedException)

Dumping configuration of initialized storage backend sources.
Please make sure they are correct.

Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> 
/home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4, 
HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstcluster, TYPE -> 
elasticsearch

Can anyone give me an idea of what I need to fix this issue?  Here is 

# PredictionIO Main Configuration
#
# This section controls core behavior of PredictionIO. It is very likely that
# you need to change these to fit your site.

# SPARK_HOME: Apache Spark is a hard dependency and must be configured.
# SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6

POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar

# ES_CONF_DIR: You must configure this if you have advanced configuration for
#              your Elasticsearch setup.
ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-1.4.4/conf

# HADOOP_CONF_DIR: You must configure this if you intend to run PredictionIO
#                  with Hadoop 2.
HADOOP_CONF_DIR=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6/conf

# HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO
#                 with HBase on a remote cluster.
HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf

# Filesystem paths where PredictionIO uses as block storage.
PIO_FS_BASEDIR=$HOME/.pio_store
PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp

# PredictionIO Storage Configuration
#
# This section controls programs that make use of PredictionIO's built-in
# storage facilities. Default values are shown below.
#
# For more information on storage configuration please refer to
# http://predictionio.incubator.apache.org/system/anotherdatastore/

# Storage Repositories

# Default is to use PostgreSQL
PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH

PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE

PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS

# Storage Data Sources

# PostgreSQL Default Settings
# Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL
# Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
# PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
# PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
# PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio

# MySQL Example
# PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
# PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
# PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
# PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio

# Elasticsearch Example
# PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
# PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
# PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
# PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
# PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2
# Optional basic HTTP auth
# PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name
# 

Re: Unable to connect to all storage backends successfully

[Pat Ferrel]

meaning is “firstcluster” the cluster name in your Elasticsearch configuration?


On Sep 19, 2017, at 8:54 PM, Vaghawan Ojha  wrote:

I think the problem is with Elasticsearch, are you sure the cluster exists in 
elasticsearch configuration? 

On Wed, Sep 20, 2017 at 8:17 AM, Jim Miller > wrote:
Hi,

I’m using PredictionIO 0.12.0-incubating with ElasticSearch and Hbase:
PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4
PredictionIO-0.12.0-incubating/vendors/hbase-1.0.0
PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6

All starts with no errors but with pio status I get:

[INFO] [Management$] Inspecting PredictionIO...
[INFO] [Management$] PredictionIO 0.12.0-incubating is installed at 
/home/vagrant/pio/PredictionIO-0.12.0-incubating
[INFO] [Management$] Inspecting Apache Spark...
[INFO] [Management$] Apache Spark is installed at 
/home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/spark-1.5.1-bin-hadoop2.6
[INFO] [Management$] Apache Spark 1.5.1 detected (meets minimum requirement of 
1.3.0)
[INFO] [Management$] Inspecting storage backend connections...
[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
[ERROR] [Management$] Unable to connect to all storage backends successfully.
The following shows the error message from the storage backend.

Connection closed 
(org.apache.predictionio.shaded.org.apache.http.ConnectionClosedException)

Dumping configuration of initialized storage backend sources.
Please make sure they are correct.

Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> 
/home/vagrant/pio/PredictionIO-0.12.0-incubating/vendors/elasticsearch-1.4.4, 
HOSTS -> localhost, PORTS -> 9300, CLUSTERNAME -> firstcluster, TYPE -> 
elasticsearch

Can anyone give me an idea of what I need to fix this issue?  Here is 

# PredictionIO Main Configuration
#
# This section controls core behavior of PredictionIO. It is very likely that
# you need to change these to fit your site.

# SPARK_HOME: Apache Spark is a hard dependency and must be configured.
# SPARK_HOME=$PIO_HOME/vendors/spark-2.0.2-bin-hadoop2.7
SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6

POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar

# ES_CONF_DIR: You must configure this if you have advanced configuration for
#  your Elasticsearch setup.
ES_CONF_DIR=$PIO_HOME/vendors/elasticsearch-1.4.4/conf

# HADOOP_CONF_DIR: You must configure this if you intend to run PredictionIO
#  with Hadoop 2.
HADOOP_CONF_DIR=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6/conf

# HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO
# with HBase on a remote cluster.
HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf

# Filesystem paths where PredictionIO uses as block storage.
PIO_FS_BASEDIR=$HOME/.pio_store
PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp

# PredictionIO Storage Configuration
#
# This section controls programs that make use of PredictionIO's built-in
# storage facilities. Default values are shown below.
#
# For more information on storage configuration please refer to
# http://predictionio.incubator.apache.org/system/anotherdatastore/ 


# Storage Repositories

# Default is to use PostgreSQL
PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH

PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE

PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS

# Storage Data Sources

# PostgreSQL Default Settings
# Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL
# Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
# PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
# PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
# PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio

# MySQL Example
# PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
# PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
# PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
# PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio

# Elasticsearch Example
# PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
# PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
# PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
# PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
# PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch-5.5.2
# Optional basic HTTP auth
# PIO_STORAGE_SOURCES_ELASTICSEARCH_USERNAME=my-name
# PIO_STORAGE_SOURCES_ELASTICSEARCH_PASSWORD=my-secret
# Elasticsearch 1.x Example
PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=firstcluster