Re: How to access Spark Context in predict?

2016-09-27 Thread Hasan Can Saral
Hi Kenneth & Donald, That was really clarifying, thank you. I really appreciate it. So now I know that; 1- I should use LEventStore and query HBase without sc and with the smallest processing as possible in predict, 2- In this case I don't have to extend PersistentModel, since I will not need sc

Re: delay of engines

2016-09-27 Thread Georg Heiler
indeed. I am mostly interested in 2) Kenneth Chan schrieb am Di., 27. Sep. 2016 um 09:22 Uhr: > just want to clarify when you mix ""evaluation / prediction" > 1. "evaluation" means evaluating the performance of the model > 2. "prediction" means calculate the prediction

Re: delay of engines

2016-09-27 Thread Georg Heiler
For me, the latency of model evaluation is more important than training latency. This holds true for retraining / model updates as well. I would say that the "evaluation / prediction" latency is the most critical one. Your point regarding 3) is very interesting for me. I have 2 types of data: