Re:deploy predictionio with spark standalone cluster

[王斌斌]

Instead of using a standalone cluster installed on another docker container, I 
start a cluster with the spark on the same container with train server, 
it's successfully.
I don't understand why it happened. I can't use a local spark, it's very 
strange.





At 2018-05-10 11:12:04, "王斌斌"  wrote:

https://stackoverflow.com/questions/50256449/deploy-predictionio-with-spark-standalone-cluster



I use the official Recommendation as a test. I did these steps successfully:

event server installed in a docker container.(successfully)
config eventdata, metadata and all things are stored in mysql.(successfully)
train & deploy server in another docker container.(successfully)
spark standalone cluster.(successfully)
create new app.(successfully)
import enough eventdata.(successfully)

When I train and deploy as follows, it's ok as the docs described :

pio train
pio deploy


But when I use spark cluster, train and deploy as follows, train is ok(new 
model has been stored in mysql), but deploy isn't success.

pio train -v engine.json -- --master spark://predictionspark:7077 
--executor-memory 2G --driver-memory 2G --total-executor-cores 1
pio deploy -v engine.json --feedback --event-server-ip predictionevent 
--event-server-port 7070 --accesskey 
Th7k5gE5yEu9ZdTdM6KdAj0InDrLNJQ1U3qEBy7dbMnYgTxWx5ALNAa2hKjqaHSK -- --master 
spark://predictionspark:7077 --executor-memory 2G --driver-memory 2G 
--total-executor-cores 1


deploy ERROR log:

...
flb_flb_1 | 2018-05-09T09:56:20.410043835Z [INFO] [Engine] Using persisted model
flb_flb_1 | 2018-05-09T09:56:20.411705255Z [INFO] [Engine] Custom-persisted 
model detected for algorithm org.example.recommendation.ALSAlgorithm
flb_flb_1 | 2018-05-09T09:56:21.263570490Z [ERROR] [OneForOneStrategy] empty 
collection


I don't know why.





 





 

deploy predictionio with spark standalone cluster

[王斌斌]

https://stackoverflow.com/questions/50256449/deploy-predictionio-with-spark-standalone-cluster



I use the official Recommendation as a test. I did these steps successfully:

event server installed in a docker container.(successfully)
config eventdata, metadata and all things are stored in mysql.(successfully)
train & deploy server in another docker container.(successfully)
spark standalone cluster.(successfully)
create new app.(successfully)
import enough eventdata.(successfully)

When I train and deploy as follows, it's ok as the docs described :

pio train
pio deploy


But when I use spark cluster, train and deploy as follows, train is ok(new 
model has been stored in mysql), but deploy isn't success.

pio train -v engine.json -- --master spark://predictionspark:7077 
--executor-memory 2G --driver-memory 2G --total-executor-cores 1
pio deploy -v engine.json --feedback --event-server-ip predictionevent 
--event-server-port 7070 --accesskey 
Th7k5gE5yEu9ZdTdM6KdAj0InDrLNJQ1U3qEBy7dbMnYgTxWx5ALNAa2hKjqaHSK -- --master 
spark://predictionspark:7077 --executor-memory 2G --driver-memory 2G 
--total-executor-cores 1


deploy ERROR log:

...
flb_flb_1 | 2018-05-09T09:56:20.410043835Z [INFO] [Engine] Using persisted model
flb_flb_1 | 2018-05-09T09:56:20.411705255Z [INFO] [Engine] Custom-persisted 
model detected for algorithm org.example.recommendation.ALSAlgorithm
flb_flb_1 | 2018-05-09T09:56:21.263570490Z [ERROR] [OneForOneStrategy] empty 
collection


I don't know why.





 

Re: UR: build/train/deploy once & querying for 3 use cases

[Pat Ferrel]

Why do you want to throw away user behavior in making recommendations? The
lift you get in purchases will be less.

There is a use case for this when you are making recommendations basically
inside a session where the user is browsing/viewing things on a hunt for
something. In this case you would want to make recs using the user history
of views but you have to build a model of purchase as the primary indicator
or you won’t get purchase recommendations and believe me recommending views
is a road to bad results. People view many things they do not buy, putting
only view behavior that lead to purchases in the model. So create a model
with purchase as the primary indicator and view as the secondary.

Once you have the model use only the user’s session viewing history in the
as the Elasticsearch query.

This is a feature on our list.


From: gerasimos xydas 

Reply: user@predictionio.apache.org 

Date: May 9, 2018 at 6:20:46 AM
To: user@predictionio.apache.org 

Subject:  UR: build/train/deploy once & querying for 3 use cases

Hello everybody,

We are experimenting with the Universal Recommender to provide
recommendations for the 3 distinct use cases below:

- Get a product recommendation based on product views
- Get a product recommendation based on product purchases
- Get a product recommendation based on previous purchases and views (i.e.
users who viewed this bought that)

The event server is fed from a single app with two types of events: "view"
and "purchase".

1. How should we customize the query to fetch results for each separate
case?
2. Is it feasible to build/train/deploy only once, and query for all 3 use
cases?


Best Regards,
Gerasimos

Re: error in pio train E-Commerce Recommender

[Marco Goldin]

thanks!

2018-05-09 14:01 GMT+02:00 Rajesh Jangid :

> Hi Marco,
>   Yes, I came across the same error a little while ago and it seems there
> should be at least one viewEvent in data to train recommender.
>
> On Wed, May 9, 2018 at 5:18 PM, Marco Goldin 
> wrote:
>
>> hi, i'm new to PredictionIO and i'm trying to use the E-Commerce
>> Recommender with some sample data, but when i launch pio train i get this
>> error which cause the train to abort:
>>
>> "Exception in thread "main" java.lang.IllegalArgumentException:
>> requirement failed: viewEvents in PreparedData cannot be empty. Please
>> check if DataSource generates TrainingData and Preprator generates
>> PreparedData correctly."
>>
>> ​i imported events from input file with required  json format:
>>
>> {"entityType": "user", "targetEntityType": "item", "event": "buy",
>> "entityId": "341785", "eventTime": "2013-01-02T18:42:15.716Z",
>> "targetEntityId": "150020361"}
>>
>>  ​does it mean that there must be at least some "view" events in the data
>> otherwise the recommender cannot train properly?​
>>
>> thanks
>> Marco
>>
>
>

Re: error in pio train E-Commerce Recommender

[Marco Goldin]

oh, this was exactly what i didn't guess.
thanks for helping Rajesh!

2018-05-09 14:01 GMT+02:00 Rajesh Jangid :

> Hi Marco,
>   Yes, I came across the same error a little while ago and it seems there
> should be at least one viewEvent in data to train recommender.
>
> On Wed, May 9, 2018 at 5:18 PM, Marco Goldin 
> wrote:
>
>> hi, i'm new to PredictionIO and i'm trying to use the E-Commerce
>> Recommender with some sample data, but when i launch pio train i get this
>> error which cause the train to abort:
>>
>> "Exception in thread "main" java.lang.IllegalArgumentException:
>> requirement failed: viewEvents in PreparedData cannot be empty. Please
>> check if DataSource generates TrainingData and Preprator generates
>> PreparedData correctly."
>>
>> ​i imported events from input file with required  json format:
>>
>> {"entityType": "user", "targetEntityType": "item", "event": "buy",
>> "entityId": "341785", "eventTime": "2013-01-02T18:42:15.716Z",
>> "targetEntityId": "150020361"}
>>
>>  ​does it mean that there must be at least some "view" events in the data
>> otherwise the recommender cannot train properly?​
>>
>> thanks
>> Marco
>>
>
>

error in pio train E-Commerce Recommender

[Marco Goldin]

hi, i'm new to PredictionIO and i'm trying to use the E-Commerce
Recommender with some sample data, but when i launch pio train i get this
error which cause the train to abort:

"Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: viewEvents in PreparedData cannot be empty. Please check if
DataSource generates TrainingData and Preprator generates PreparedData
correctly."

​i imported events from input file with required  json format:

{"entityType": "user", "targetEntityType": "item", "event": "buy",
"entityId": "341785", "eventTime": "2013-01-02T18:42:15.716Z",
"targetEntityId": "150020361"}

 ​does it mean that there must be at least some "view" events in the data
otherwise the recommender cannot train properly?​

thanks
Marco