Thank you Vincent and Jorn for your inputs, much appreciated.
Our web-app already has a scheduler mechanism and other jobs are already
running in the system. Would you still prefer to decouple model training in
a separate scheduling tool outside of our web-app JVM?
We are using test Data for now a
Decoupling the web app from Spark backend is recommended. Training the
model can be launched in the background via a scheduling tool. Inferring
the model with Spark in interactive mode s not a good option as it will do
it for unitary data and Spark is better in using large dataset. The
original pur
Depending on your model size you can store it as PFA or PMML and run the
prediction in Java. For larger models you will need a custom solution ,
potentially using a spark thrift Server/spark job server/Livy and a cache to
store predictions that have been already calculated (eg based on previous
On Mon, Oct 1, 2018 at 12:18 PM Girish Vasmatkar <
girish.vasmat...@hotwaxsystems.com> wrote:
> Hi All
>
> We are very early into our Spark days so the following may sound like a
> novice question :) I will try to keep this as short as possible.
>
> We are trying to use Spark to introduce a recomm
All
Can someone please shed some light on the above query? Any help is greatly
appreciated.
Thanks,
Girish Vasmatkar
HotWax Systems
On Thu, Oct 4, 2018 at 10:25 AM Girish Vasmatkar <
girish.vasmat...@hotwaxsystems.com> wrote:
>
>
> On Mon, Oct 1, 2018 at 12:18 PM Girish Vasmatkar <
> girish.va
Hi All
We are very early into our Spark days so the following may sound like a
novice question :) I will try to keep this as short as possible.
We are trying to use Spark to introduce a recommendation engine that can be
used to provide product recommendations and need help on some design
decision
