It actually doesn't look like you use spark context within the predict method itself...
Try Removing the spark context ref from the method Params and also the (sc) at the end of the predict method. On Thu, Oct 5, 2017 at 10:13 PM Shane Johnson <shanewaldenjohn...@gmail.com> wrote: > Thanks Daniel. I had the P2Algorithm working before I had to query the > EventStore within the predict method. Do you think this is still the issue > with the context that I had it working before attempting to add the > SparkContext? > > *Shane Johnson | 801.360.3350* > LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook > <https://www.facebook.com/shane.johnson.71653> > > On Thu, Oct 5, 2017 at 3:07 PM, Daniel O' Shaughnessy < > danieljamesda...@gmail.com> wrote: > >> Hi Shane, >> >> Your RFAlgorithm class needs to use PAlgorithm instead of P2Algorithm. >> You then need to write some code to save and load your model and spark >> context etc. >> >> There should be examples of this on the predictionio site somewhere >> >> On Thu, Oct 5, 2017 at 10:00 PM Shane Johnson < >> shanewaldenjohn...@gmail.com> wrote: >> >>> Hi team, >>> >>> Can someone guide how I can add SparkContext into the predict method. I >>> am using unique ids that I gather from Query and pulling back additional >>> attributes from the PEventStore and am getting an error that "sc" cannot be >>> found. When I add SparkContext to the method I get the following error. >>> >>> Can anyone provide direction here? >>> >>> Thank you >>> >>> Adding SparkContext eliminates the first error but produces another. >>> >>> def predict(sc: SparkContext, model: RFModel, query: Query): >>> PredictedResult = { >>> >>> val featureIndex = model.featureIndex >>> val featureCategoricalIntMap = model.featureCategoricalIntMap >>> >>> >>> val responses: List[PredictionResponses] = query.predictionRequests >>> .map { >>> Predictions => >>> >>> val oppPost = PEventStore.aggregateProperties( >>> appName = sys.env("PIO_EVENTSERVER_APP_NAME"), >>> entityType = "Opportunity" >>> )(sc) >>> >>> >>> First error when sc: SparkContext is not added to the method parameters: >>> >>> not found: value sc >>> [INFO] [Engine$] [error] )(sc) >>> [INFO] [Engine$] [error] ^ >>> >>> >>> Error after adding SparkContext: >>> >>> class RFAlgorithm needs to be abstract, since method predict in class >>> P2LAlgorithm of type (model: org.template.liftscori >>> ng.RFModel, query: org.template.liftscoring.Query)org.template >>> .liftscoring.PredictedResult is not defined >>> [INFO] [Engine$] [error] class RFAlgorithm(val ap: RFAlgorithmParams) >>> [INFO] [Engine$] [error] ^ >>> [INFO] [Engine$] [error] one error found >>> [INFO] [Engine$] [error] (compile:compileIncremental) Compilation failed >>> [INFO] [Engine$] [error] Total time: 6 s, completed Oct 5, 2017 2:44:51 >>> PM >>> >>> *Shane Johnson | 801.360.3350 <(801)%20360-3350>* >>> LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook >>> <https://www.facebook.com/shane.johnson.71653> >>> >> >
