Re: [DISCUSS] Dissolve Apache PredictionIO PMC and move project to the Attic

2020-08-31 Thread Pat Ferrel
To try to keep this on-subject I’ll say that I’ve been working on what I once saw as a next-gen PIO. It is ASL 2, and has 2 engines that ran in PIO — most notably the Universal Recommender. We offered to make the Harness project part of PIO a couple years back but didn’t get much interest. It

Re: PredictionIO ASF Board Report for Mar 2020

2020-03-19 Thread Pat Ferrel
PredictionIO is scalable BY SCALING ITS SUB-SERVICES. Running on a single machine sounds like no scaling has been executed or even planned. How do you scale ANY system? 1) vertical scaling: make the instance larger with more cores, more disk, and most importantly more memory. Increase whatever

Re: JAVA_HOME is not set

2019-07-03 Thread Pat Ferrel
Oops, should have said: "I may have missed something but I don’t recall PIO being released by Apache as an ASF maintained container/image release artifact." From: Pat Ferrel Reply: user@predictionio.apache.org Date: July 3, 2019 at 11:16:43 AM To: Wei Chen , d...@predictionio.

Re: JAVA_HOME is not set

2019-07-03 Thread Pat Ferrel
BTW the container you use is supported by the container author, if at all. I may have missed something but I don’t recall PIO being released by Apache as an ASF maintained release artifact. I wish ASF projects would publish Docker Images made for real system integration, but IIRC PIO does not.

Re: new install help

2019-04-15 Thread Pat Ferrel
Most people running on a Windows machine use a VM running Linux. You will run into constant issues if you go down another road with something like cygwin, so avoid the headache. From: Steve Pruitt Reply: user@predictionio.apache.org Date: April 15, 2019 at 10:59:09 AM To:

Re: universal recommender version

2018-11-27 Thread Pat Ferrel
There is a tag v0.7.3 and yes it is in master: https://github.com/actionml/universal-recommender/tree/v0.7.3 From: Marco Goldin Reply: user@predictionio.apache.org Date: November 20, 2018 at 6:56:39 AM To: user@predictionio.apache.org , gyar...@griddynamics.com Subject:  Re: universal

Re: PIO train issue

2018-08-29 Thread Pat Ferrel
Assuming your are using the UR… I don’t know how many times this has been caused by a misspelling of eventNames in engine.json but assume you have checked that. The fail-safe way to check is to `pio export` your data and check it against your engine.json. BTW `pio status` does not even try to

Re: Distinct recommendation from "random" backfill?

2018-08-28 Thread Pat Ferrel
The random ranking is assigned after every `pio train` so if you have not trained in-between, they will be the same. Random is not really meant to do what you are using it for, it is meant to surface items with no data—no primary events. This will allow some to get real events and be recommended

Re: PredictionIO spark deployment in Production

2018-08-07 Thread Pat Ferrel
Oh and no it does not need a new context for every query, only for the deploy. From: Pat Ferrel Date: August 7, 2018 at 10:00:49 AM To: Ulavapalle Meghamala Cc: user@predictionio.apache.org , actionml-user Subject: Re: PredictionIO spark deployment in Production The answers to your

Re: PredictionIO spark deployment in Production

2018-08-07 Thread Pat Ferrel
into Elasticsearch for serving independently scalable queries. I always advise you keep Spark out of serving for the reasons mentioned above. From: Ulavapalle Meghamala Date: August 7, 2018 at 9:27:46 AM To: Pat Ferrel Cc: user@predictionio.apache.org , actionml-user Subject: Re

Re: PredictionIO spark deployment in Production

2018-08-07 Thread Pat Ferrel
PIO is designed to use Spark in train and deploy. But the Universal Recommender removes the need for Spark to make predictions. This IMO is a key to use Spark well—remove it from serving results. PIO creates a Spark context to launch the `pio deploy' driver but Spark is never used and the context

Re: Straw poll: deprecating Scala 2.10 and Spark 1.x support

2018-08-02 Thread Pat Ferrel
+1 From: takako shimamoto Reply: user@predictionio.apache.org Date: August 2, 2018 at 2:55:49 AM To: d...@predictionio.apache.org , user@predictionio.apache.org Subject: Straw poll: deprecating Scala 2.10 and Spark 1.x support Hi all, We're considering deprecating Scala 2.10 and Spark

Re: 2 pio servers with 1 event server

2018-08-02 Thread Pat Ferrel
What template? From: Sami Serbey Reply: user@predictionio.apache.org Date: August 2, 2018 at 9:08:05 AM To: user@predictionio.apache.org Subject: 2 pio servers with 1 event server Greetings, I am trying to run 2 pio servers on different ports where each server have his own app. When I

Re: [actionml/universal-recommender] Boosting categories only shows one category type (#55)

2018-07-06 Thread Pat Ferrel
Please read the docs. There is no need to $set users since they are attached to usage events and can be detected automatically. In fact "$set"ting them is ignored. There are no properties of users that are not calculated based on named “indicators’, which can be profile type things. Fot this

Re: Digging into UR algorithm

2018-07-02 Thread Pat Ferrel
-id, "searched-for”, search-term) This as a secondary event has proven to be quite useful in at least one dataset I’ve seen. From: Pat Ferrel Reply: Pat Ferrel Date: July 2, 2018 at 12:18:16 PM To: user@predictionio.apache.org , Sami Serbey Cc: actionml-user Subject: Re: Digging in

Re: Digging into UR algorithm

2018-07-02 Thread Pat Ferrel
The only requirement is that someone performed the primary event on A and the secondary event is correlated to that primary event. the UR can recommend to a user who has only performed the secondary event on B as long as that is in the model. Makes no difference what subset of events the user has

Re: a question about a high availability of Elasticsearch cluster

2018-06-22 Thread Pat Ferrel
This should work with any node down. Elasticsearch should elect a new master. What version of PIO are you using? PIO and the UR changed the client from the transport client to the RET client in 0.12.0, which is why you are using port 9200. Do all PIO functions work correctly like: - pio app

Re: UR trending ranking as separate process

2018-06-20 Thread Pat Ferrel
 user@predictionio.apache.org Date: June 20, 2018 at 10:25:53 AM To: user@predictionio.apache.org , Pat Ferrel Cc: user@predictionio.apache.org Subject:  Re: UR trending ranking as separate process Hi George, I didn't get your question but I think I am missing something. So you're using the Univ

Re: UR trending ranking as separate process

2018-06-20 Thread Pat Ferrel
No the trending algorithm is meant to look at something like trends over 2 days. This is because it looks at 2 buckets of conversion frequencies and if you cut them smaller than a day you will have so much bias due to daily variations that the trends will be invalid. In other words the ups and

Re: java.util.NoSuchElementException: head of empty list when running train

2018-06-19 Thread Pat Ferrel
PIO_STORAGE_SOURCES_HBASE_HOME=/usr/local/hbase Thanks, Anuj Kumar On Tue, Jun 19, 2018 at 9:16 PM Pat Ferrel wrote: > Can you show me where on the AML site it says to store models in HDFS, it > should not say that? I think that may be from the PIO site so you should > ignore it. > > Can

Re: java.util.NoSuchElementException: head of empty list when running train

2018-06-19 Thread Pat Ferrel
; based backfill, must add eventsNames", > > "name": "ur", > > "params": { > > "appName": "np", > > "indexName": "np", > > "typeName": "items", > > "blacklistEvents": [],

Re: Few Queries Regarding the Recommendation Template

2018-06-13 Thread Pat Ferrel
te gets it wrong. From: KRISH MEHTA Reply: KRISH MEHTA Date: June 13, 2018 at 2:19:17 PM To: Pat Ferrel Subject: Re: Few Queries Regarding the Recommendation Template I Understand but if I just want the likes, dislikes and views then I can combine the algorithms right? Given in

Re: True Negative - ROC Curve

2018-06-12 Thread Pat Ferrel
We do not use these for recommenders. The precision rate is low when the lift in your KPI like sales is relatively high. This is not like classification. We use MAP@k with increasing values of k. This should yield a diminishing mean average precision chart with increasing k. This tells you 2

Re: Regarding Real-Time Prediction

2018-06-11 Thread Pat Ferrel
Actually if you are using the Universal Recommender you only need to deploy once as long as the engine.json does not change. The hot swap happens as @Digambar says and there is literally no downtime. If you are using any of the other recommenders you do have to re-deploy after every train but

Re: UR template minimum event number to recommend

2018-06-04 Thread Pat Ferrel
No but we have 2 ways to handle this situation automatically and you can tell if recommendations are not from personal user history. 1. when there is not enough user history to recommend, we fill in the lower ranking recommendations with popular, trending, or hot items. Not completely

Re: PIO 0.12.1 with HDP Spark on YARN

2018-05-29 Thread Pat Ferrel
Yarn has to be started explicitly. Usually it is part of Hadoop and is started with Hadoop. Spark only contains the client for Yarn (afaik). From: Miller, Clifford Reply: user@predictionio.apache.org Date: May 29, 2018 at 6:45:43 PM To: user@predictionio.apache.org Subject: Re: PIO

Re: Spark cluster error

2018-05-29 Thread Pat Ferrel
Sorry, what I meant was the actual spark-submit command that PIO was using. It should be in the log. What Spark version was that? I recall classpath issues with certain versions of Spark. On Thu, May 24, 2018 at 4:52 PM, Pat Ferrel wrote: > Thanks Donald, > > We have: > >

Re: pio app new failed in hbase

2018-05-29 Thread Pat Ferrel
No, this is as expected. When you run pseudo-distributed everything internally is configured as if the services were on separate machines. See clustered instructions here: http://actionml.com/docs/small_ha_cluster This is to setup 3 machines running different parts and is not really the best

Re: PIO not using HBase cluster

2018-05-25 Thread Pat Ferrel
rd.mil...@phoenix-opsgroup.com> Reply: Miller, Clifford <clifford.mil...@phoenix-opsgroup.com> <clifford.mil...@phoenix-opsgroup.com> Date: May 25, 2018 at 10:16:01 AM To: Pat Ferrel <p...@occamsmachete.com> <p...@occamsmachete.com> Cc: user@predictionio.apache.org <user@predict

Re: PIO not using HBase cluster

2018-05-25 Thread Pat Ferrel
No, you need to have HBase installed, or at least the config installed on the PIO machine. The pio-env.sh defined servers will be configured cluster operations and will be started separately from PIO. PIO then will not start hbase and try to sommunicate only, not start it. But PIO still needs

Re: Spark2 with YARN

2018-05-24 Thread Pat Ferrel
I’m having a java.lang.NoClassDefFoundError in a different context and different class. Have you tried this without Yarn? Sorry I can’t find the rest of this thread. From: Miller, Clifford Reply:

Re: Spark cluster error

2018-05-24 Thread Pat Ferrel
doop2 in the storage driver assembly. Looking at Git history it has not changed in a while. Do you have the exact classpath that has gone into your Spark cluster? On Wed, May 23, 2018 at 1:30 PM, Pat Ferrel <p...@actionml.com> wrote: > A source build did not fix the problem, has anyone r

Re: Spark cluster error

2018-05-23 Thread Pat Ferrel
ster=local but not with remote Spark master I’ve passed in the hbase-client in the --jars part of spark-submit, still fails, what am I missing? From: Pat Ferrel <p...@actionml.com> <p...@actionml.com> Reply: Pat Ferrel <p...@actionml.com> <p...@actionml.com> Date: May 23, 2018 at 8:57:32 A

Spark cluster error

2018-05-23 Thread Pat Ferrel
Same CLI works using local Spark master, but fails using remote master for a cluster due to a missing class def for protobuf used in hbase. We are using the binary dist 0.12.1. Is this known? Is there a work around? We are now trying a source build in hope the class will be put in the assembly

RE: Problem with training in yarn cluster

2018-05-23 Thread Pat Ferrel
e case where yarn is tyring to findout pio.log file on hdfs cluster. You can try "--master yarn --deploy-mode client ". you need to pass this configuration with pio train e.g., pio train -- --master yarn --deploy-mode client Thanks and Regards Ambuj Sharma Sunrise may late, B

RE: Problem with training in yarn cluster

2018-05-22 Thread Pat Ferrel
arbitrary Spark params exactly as you would to spark-submit on the pio command line. The double dash separates PIO and Spark params. From: Pat Ferrel <p...@occamsmachete.com> <p...@occamsmachete.com> Reply: user@predictionio.apache.org <user@predictionio.apache.org> <user@pr

RE: Problem with training in yarn cluster

2018-05-22 Thread Pat Ferrel
What is the command line for `pio train …` Specifically are you using yarn-cluster mode? This causes the driver code, which is a PIO process, to be executed on an executor. Special setup is required for this. From: Wojciech Kowalski Reply: user@predictionio.apache.org

Re: UR: build/train/deploy once & querying for 3 use cases

2018-05-11 Thread Pat Ferrel
BTW The Universal Recommender has it’s own community support group here: https://groups.google.com/forum/#!forum/actionml-user From: Pat Ferrel <p...@occamsmachete.com> <p...@occamsmachete.com> Reply: user@predictionio.apache.org <user@predictionio.apache.org> <user@predicti

Re: UR: build/train/deploy once & querying for 3 use cases

2018-05-11 Thread Pat Ferrel
and “ItemBias” on the query > do not have any effect on the result. > > 5.Is it feasible to build/train/deploy only once, and query for > all 3 use cases? > > > 6. How to make queries towards the different Apps because there is > no any obvious way in the query para

Re: UR evaluation

2018-05-10 Thread Pat Ferrel
Exactly, ranking is the only task of a recommender. Precision is not automatically good at that but something like MAP@k is. From: Marco Goldin <markomar...@gmail.com> <markomar...@gmail.com> Date: May 10, 2018 at 10:09:22 PM To: Pat Ferrel <p...@occamsmachete.com> <p...@

Re: UR evaluation

2018-05-10 Thread Pat Ferrel
3 AM To: Pat Ferrel <p...@occamsmachete.com> Cc: user@predictionio.apache.org <user@predictionio.apache.org> Subject:  Re: UR evaluation thank you very much, i didn't see this tool, i'll definitely try it. Clearly better to have such a specific instrument. 2018-05-10 18:36

Re: UR evaluation

2018-05-10 Thread Pat Ferrel
You can if you want but we have external tools for the UR that are much more flexible. The UR has tuning that can’t really be covered by the built in API. https://github.com/actionml/ur-analysis-tools They do MAP@k as well as creating a bunch of other metrics and comparing different types of input

Re: UR: build/train/deploy once & querying for 3 use cases

2018-05-09 Thread Pat Ferrel
Why do you want to throw away user behavior in making recommendations? The lift you get in purchases will be less. There is a use case for this when you are making recommendations basically inside a session where the user is browsing/viewing things on a hunt for something. In this case you would

Re: Info / resources for scaling PIO?

2018-04-24 Thread Pat Ferrel
PIO is based on the architecture of Spark, which uses HDFS. HBase also uses HDFS. Scaling these are quite well documented on the web. Scaling PIO is the same as scaling all it’s services. It is unlikely you’ll need it but you can also have more than one PIO server behind a load balancer. Don’t

Re: pio deploy without spark context

2018-04-14 Thread Pat Ferrel
The need for Spark at query time depends on the engine. Which are you using? The Universal Recommender, which I maintain, does not require Spark for queries but uses PIO. We simply don’t use the Spark context so it is ignored. To make PIO work you need to have the Spark code accessible but that

Re: Hbase issue

2018-04-13 Thread Pat Ferrel
This may seem unhelpful now but for others it might be useful to mention some minimum PIO in production best practices: 1) PIO should IMO never be run in production on a single node. When all services share the same memory, cpu, and disk, it is very difficult to find the root cause to a

Re: how to set engine-variant in intellij idea

2018-04-10 Thread Pat Ferrel
There are instructions for using Intellij but, I wrote the last version, I apologize that I can’t make them work anymore. If you get them to work you would be doing the community a great service by telling us how or editing the instructions. http://predictionio.apache.org/resources/intellij/

Re: Unclear problem with using S3 as a storage data source

2018-03-29 Thread Pat Ferrel
: user@predictionio.apache.org <user@predictionio.apache.org> Date: March 29, 2018 at 6:19:58 AM To: Pat Ferrel <p...@occamsmachete.com> Cc: user@predictionio.apache.org <user@predictionio.apache.org> Subject:  Re: Unclear problem with using S3 as a storage data source Sorry

Re: Unclear problem with using S3 as a storage data source

2018-03-28 Thread Pat Ferrel
: Dave Novelli <d...@ultravioletanalytics.com> <d...@ultravioletanalytics.com> Date: March 28, 2018 at 12:13:12 PM To: Pat Ferrel <p...@occamsmachete.com> <p...@occamsmachete.com> Cc: user@predictionio.apache.org <user@predictionio.apache.org> <user@predictionio.

Re: Error when training The Universal Recommender 0.7.0 with PredictionIO 0.12.0-incubating

2018-03-27 Thread Pat Ferrel
Pio build requires that ES hosts are known to Spark, which write the model to ES. You can pass these in on the `pio train` command line: pio train … -- --conf spark.es.nodes=“node1,node2,node3” notice no spaces in the quoted list of hosts, also notice the double dash, which separates pio

Re: UR 0.7.0 - problem with training

2018-03-08 Thread Pat Ferrel
BTW I think you may have to push setting on the cli by adding “spark” to the beginning of the key name: *pio train -- --conf spark.es.nodes=**“**localhost" --driver-memory 8g --executor-memory 8g* From: Pat Ferrel <p...@occamsmachete.com> <p...@occamsmachete.co

Re: UR 0.7.0 - problem with training

2018-03-08 Thread Pat Ferrel
es.nodes is supposed to be a string with hostnames separated by commas. Depending on how your containers are set to communicate with the outside world (Docker networking or port mapping) you may also need to set the port, which is 9200 by default. If your container is using port mapping and maps

Re: Dynamically change parameter list

2018-02-12 Thread Pat Ferrel
meters in this step. I'll let you know if it works. Thanks, Tihomie On Feb 12, 2018 23:34, "Pat Ferrel" <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: This is an interesting question. As we make more mature full featured engines they will begin to employ hyper

Re: Dynamically change parameter list

2018-02-12 Thread Pat Ferrel
This is an interesting question. As we make more mature full featured engines they will begin to employ hyper parameter search techniques or reinforcement params. This means that there is a new stage in the workflow or a feedback loop not already accounted for. Short answer is no, unless you want

Re: pio train on Amazon EMR

2018-02-05 Thread Pat Ferrel
I agree, we looked at using EMR and found that we liked some custom Terraform + Docker much better. The existing EMR defined by AWS requires refactoring PIO or using it in yarn’s cluster mode. EMR is not meant to host any application code except what is sent into Spark in serialized form.

Re: Frequent Pattern Mining - No engine found. Your build might have failed. Aborting.

2018-02-01 Thread Pat Ferrel
This list is for support of ActionML products, not general PIO support. You can get that on the Apache PIO user mailing list, where I have forwarded this question. Several uses of FPM are supported by the Universal Recommender, such as Shopping cart recommendations. That is a template we

Re: PIO error

2018-01-23 Thread Pat Ferrel
Unfortunately I can’t possibly guess without more information. What do the logs say when pio cannot be started? Are all these pio instances separate, not in a cluster? In other words does each pio server have all necessary services running on them? I assume none is sleeping like a laptop does? I

Re: Prediction IO install failed in Linux

2018-01-23 Thread Pat Ferrel
This would be very difficult to do. Even if you used a machine connected to the internet to download things like pio, spark, etc. the very build tools used (sbt) expect to be able to get code from various repositories on the internet. To build templates would further complicate this since each

Re: Need Help Setting up prediction IO

2018-01-17 Thread Pat Ferrel
ng Pio for now, and everything is working great. Thanks for the support though. Few question- 1.Does Pio latest support Mongodb or NoSQL? 2.Memory uses by Pio, Is there any max memory limit set, If need be can it be set? Thanks Rajesh On Jan 11, 2018 10:25 PM, "Pat

The Universal Recommender v0.7.0

2018-01-17 Thread Pat Ferrel
We have been waiting to release the UR v0.7.0 for testing (done) and the release of Mahout v0.13.1 (not done) Today we have released the UR v0.7.0 anyway. This comes with: Support for PIO v0.12.0 Requires Scala 2.11 (can be converted to use Scala 2.10 but it’s a manual process) Requires

Re: Need Help Setting up prediction IO

2018-01-11 Thread Pat Ferrel
The version in the artifact built by Scala should only have the major version number so 2.10 or 2.11. PIO 0.10.0 needs 2.10. Where, and what variable did you set to 2.10.4? That is the problem. There will never be a lib built for 2.10.4, it will always be 2.10. On Jan 11, 2018, at 5:15 AM,

Re: Using Dataframe API vs. RDD API?

2018-01-05 Thread Pat Ferrel
ike: val RDD_2_DF = sqlContext.createDataFrame(yourRDD).toDF("col1", "col2") On Thu, 4 Jan 2018 at 23:10 Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: Actually there are libs that will read DFs from HBase https://svn.apache.org/repos/asf

Re: Using Dataframe API vs. RDD API?

2018-01-04 Thread Pat Ferrel
ecause the schema of the EventStore is not guaranteed to remain as-is. The safest way is to translate or get DFs integrated to PIO. I think there is an existing Jira that request Spark ML support, which assumes DFs. On Jan 4, 2018, at 12:25 PM, Pat Ferrel <p...@occamsmachete.com> wrote: Funny yo

Re: Using Dataframe API vs. RDD API?

2018-01-04 Thread Pat Ferrel
Funny you should ask this. Yes, we are working on a DF based Universal Recommender but you have to convert the RDD into a DF since PIO does not read out data in the form of a DF (yet). This is a fairly simple step of maybe one line of code but would be better supported in PIO itself. The issue

Re: Error: "unable to undeploy"

2018-01-03 Thread Pat Ferrel
The UR does not require more than one deploy (assuming the server runs forever). Retraining the UR automatically re-deploys the new model. All other Engines afaik do require retrain-redeploy. Users should be aware that PIO is a framework that provides no ML function whatsoever. It supports a

Re: App still returns results after pio app data-delete

2018-01-02 Thread Pat Ferrel
BTW there is a new Chrome extension that lets you browse ES and create any JSON query. Just found it myself after Sense stopped working in Chrome. Try ElasticSearch Head, found in the Chrome store. On Jan 2, 2018, at 9:53 AM, Pat Ferrel <p...@occamsmachete.com> wrote: Have a look at

Re: App still returns results after pio app data-delete

2018-01-02 Thread Pat Ferrel
u want to delete. On Jan 2, 2018, at 12:22 AM, Noelia Osés Fernández <no...@vicomtech.org> wrote: Thanks for the explanation! How do I delete the ES index? is it just DELETE /my_index_name? Happy New Year! On 22 December 2017 at 19:42, Pat Ferrel <p...@occamsmachete.com <mailto:p...

Re: Recommendation return score more than 5

2017-12-22 Thread Pat Ferrel
templates/recommendation/quickstart/> 2017-12-22 22:12 GMT+03:00 Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>>: Implicit means you assign a score to the event based on your own guess. Explicit uses ratings the user makes. One score is a guess by you (like a 4 for buy) a

Re: How to import item properties dynamically?

2017-12-22 Thread Pat Ferrel
The properties go into the Event Store immediately but you have to train to get them into the model, this assuming your template support item properties. If yo uare using the UR, the properties will not get into the model until the next `pio train…` On Dec 22, 2017, at 3:37 AM, Noelia Osés

Re: Recommendation return score more than 5

2017-12-22 Thread Pat Ferrel
kes an score, and the buy implicitly puts the rating at 4 (out of 5, as I think). And I still did not understand exactly where to look for me and what to correct, so that incoming and predicted estimates were on the same scale. 2017-12-19 4:10 GMT+03:00 Pat Ferrel <p...@occamsmachete

Re: Recommended Configuration

2017-12-15 Thread Pat Ferrel
That is enough for a development machine and may work if you data is relatively small but for big data clusters of CPU with a fair amount of RAM and Storage are required. The telling factor is partly how big your data is but also how is combines to form models, which will depend on which

Re: User features to tailor recs in UR queries?

2017-12-05 Thread Pat Ferrel
The User’s possible indicators of taste are encoded in the usage data. Gender and other “profile" type data can be encoded a (user-id, gender, gender-id) but this is used and a secondary indicator, not as a filter. Only item properties are used a filters for some very practical reasons. For one

Re: Log-likelihood based correlation test?

2017-11-23 Thread Pat Ferrel
ou say "compare to each item in the model", do you mean each row in PtP? On 21 November 2017 at 19:56, Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: No PtP non-zero elements have LLR calculated. The highest scores in the row are kept, or ones ab

Re: Log-likelihood based correlation test?

2017-11-20 Thread Pat Ferrel
That would be a nice feature if it doesn't exist. The way I've gotten some insight into what the model is doing is by when using PIO / UR is by inspecting the the ElasticSearch index that gets created because it has the "significant" values populated in the documents (though not the actual

Re: Error in getting Total Events in a predictionIo App

2017-11-14 Thread Pat Ferrel
You should use pio 0.12.0 if you need Elasticsearch 5.x On Nov 14, 2017, at 6:39 AM, Abhimanyu Nagrath wrote: Hi , I am new to predictionIo using version V0.11-incubating (spark - 2.6.1 , hbase - 1.2.6 , elasticsearch - 5.2.1) . Started the prediction server with

Re: Which template for predicting ratings?

2017-11-13 Thread Pat Ferrel
What I was saying is the UR can use ratings, but not predict them. Use MLlib ALS recommenders if you want to predict them for all items. On Nov 13, 2017, at 9:32 AM, Pat Ferrel <p...@occamsmachete.com> wrote: What we did in the article I attached is assume 1-2 is dislike, and 4-5 i

Re: Does PIO support [ --master yarn --deploy-mode cluster ]?

2017-11-13 Thread Pat Ferrel
yarn-cluster mode is supported but extra config needs to be set so the driver can be run on a remote machine. I have seen instructions for this on the PIO mailing list. On Nov 12, 2017, at 7:30 PM, wei li wrote: Hi Pat Thanks a lot for your advice. We are using

Re: PIO + ES5 + Universal Recommender

2017-11-08 Thread Pat Ferrel
If so, it is not said in the PredictionIO installation instructions nor on the Mahout instructions. I apologise if this is an obvious question for those familiar with the Apache projects, but for an outsider like me it helps when everything (even the most silly details) is spelled out. Thanks

Re: PIO + ES5 + Universal Recommender

2017-11-07 Thread Pat Ferrel
Very sorry, it was incorrectly set to private. Try it again. On Nov 7, 2017, at 7:26 AM, Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: https://github.com/actionml/mahout <https://github.com/actionml/mahout>

Re: PIO + ES5 + Universal Recommender

2017-11-07 Thread Pat Ferrel
sitory 'https://github.com/actionml/mahout.git/ <https://github.com/actionml/mahout.git/>' not found On 3 November 2017 at 22:27, Pat Ferrel <p...@occamsmachete.com <mailto:p...@occamsmachete.com>> wrote: The exclusion rules are working now along with the integration-test. We have

Re: Implementing cart and wishlist item events into Ecommerce recommendation template

2017-11-04 Thread Pat Ferrel
library and expect to modify the template code. There is also the Complimentary Purchase template, which does shopping carts but, from my rather prejudiced viewpoint, if you need to switch templates use one that supports every use-case you are likely to need. On Nov 4, 2017, at 9:34 AM, Pat

Re: Implementing cart and wishlist item events into Ecommerce recommendation template

2017-11-04 Thread Pat Ferrel
The Universal Recommender supports several types of “item-set” recommendations: 1) Complimentary Purchases. which are things bought with what you have in the shopping cart. This is done by training on (cart-id, “add-to-cart”, item-id) and querying with the current items in the user’s cart. 2)

Re: PIO + ES5 + Universal Recommender

2017-11-01 Thread Pat Ferrel
Ack, I hate this &^%&%^& touchbar! What I meant to say was: We have a version of the universal recommender working with PIO-0.12.0 that is ready for brave souls to test. This includes some speedups and quality of recommendation improvements, not yet documented. Known bugs: exclusion rules

PIO + ES5 + Universal Recommender

2017-11-01 Thread Pat Ferrel
We have a version working here: https://github.com/actionml/universal-recommender.git checkout 0.7.0-SNAPSHOT once you pull the repo. Known bug: exclusion rules not working. This will be fixed before release in the next few days Issues:

Templates First

2017-10-20 Thread Pat Ferrel
PredictionIO is completely useless without a Template yet we seem as a group too focused on releasing PIO without regard for Templates. This IMO must change. 90% of users will never touch the code of a template and only 1% will actually create a template. These guesses come from list questions.