Re: Is Spark right for us?

2016-03-07 Thread Laumegui Deaulobi
Thanks for your input. That 1 hour per data point actually be a problem, since sometimes we have reports with 100s of data points and need to generate 100,000 reports. So we definitely need to distribute this, but I don't know where to start with this unfortunately. On Mon, Mar 7, 2016 at 2:42

Is Spark right for us?

2016-03-06 Thread Laumegui Deaulobi
Our problem space is survey analytics. Each survey comprises a set of questions, with each question having a set of possible answers. Survey fill-out tasks are sent to users, who have until a certain date to complete it. Based on these survey fill-outs, reports need to be generated. Each