Re: Can we use spark inside a web service?

2016-03-15 Thread Andrés Ivaldi
mory. > >>>> >>> > >>>> >>> Yes, you can horizontally scale this because of the distributed > >>>> >>> nature of > >>>> >>> Spark, howev

Re: Can we use spark inside a web service?

2016-03-14 Thread Evan Chan
ample, if you're looking to scale out to 1000 concurrent >>>> >>> requests, >>>> >>> this is 1000 concurrent Spark jobs. This would require a cluster >>>> >>> with 1000 >>>> >>> cores. this is just not cost eff

Re: Can we use spark inside a web service?

2016-03-14 Thread Evan Chan
d - managing a large amount of concurrent requests. And use >>> >>> a >>> >>> database for what it's good for - storing/retrieving data. >>> >>> >>> >&

Re: Can we use spark inside a web service?

2016-03-11 Thread Andrés Ivaldi
- managing a large amount of concurrent requests. And use >>> a >>> >>> database for what it's good for - storing/retrieving data. >>> >>> >>> >>> And any serious production deployment will need failover, throttling, >>> >>> back pressu

Re: Can we use spark inside a web service?

2016-03-11 Thread Hemant Bhanawat
path. >> >>> >> >>> For the failover, throttling, back pressure, autoscaling that i >> mentioned >> >>> above, it's worth checking out the suite of Netflix OSS - particularly >> >>> Hystrix, Eureka, Zuul, Karyon, etc: http://ne

Re: Can we use spark inside a web service?

2016-03-10 Thread Nick Pentreath
Yes, really interesting discussion. It would be really interesting to compare the performance of alternative architectures. Specifically, I've found that Elasticsearch is a great option for analytic workloads - it doesn't support SQL (joins in particular), but its aggregation and arbitrary

Re: Can we use spark inside a web service?

2016-03-10 Thread Tristan Nixon
Hear, hear. That’s why I’m here :) > On Mar 10, 2016, at 7:32 PM, Chris Fregly wrote: > > Anyway, thanks for the good discussion, everyone! This is why we have these > lists, right! :)

Re: Can we use spark inside a web service?

2016-03-10 Thread Chris Fregly
se: > >>> https://github.com/cfregly/fluxcapacitor > >>> > >>> Here's a netflix Skunkworks github project that packages these up in > >>> Docker images: https://github.com/Netflix-Skunkworks/zerotodocker > >>> > >>> > >>> On

Re: Can we use spark inside a web service?

2016-03-10 Thread Evan Chan
gt;>> Docker images: https://github.com/Netflix-Skunkworks/zerotodocker >>> >>> >>> On Thu, Mar 10, 2016 at 1:40 PM, velvia.github <velvia.git...@gmail.com> >>> wrote: >>>> >>>> Hi, >>>> >>>> I

Re: Can we use spark inside a web service?

2016-03-10 Thread Teng Qiu
I just wrote a blog post which might be really useful to you -- I have just >>>>> benchmarked being able to achieve 700 queries per second in Spark. So, yes, >>>>> web speed SQL queries are definitely possible. Read my new blog post: >>>>> >>>

Re: Can we use spark inside a web service?

2016-03-10 Thread Mark Hamstra
kages these up in >>> Docker images: https://github.com/Netflix-Skunkworks/zerotodocker >>> >>> >>> On Thu, Mar 10, 2016 at 1:40 PM, velvia.github <velvia.git...@gmail.com> >>> wrote: >>> >>>> Hi, >>>> >>>

Re: Can we use spark inside a web service?

2016-03-10 Thread Chris Fregly
being able to achieve 700 queries per second in Spark. So, >>> yes, >>> web speed SQL queries are definitely possible. Read my new blog post: >>> >>> http://velvia.github.io/Spark-Concurrent-Fast-Queries/ >>> >>> and feel free to email me (at vel

Re: Can we use spark inside a web service?

2016-03-10 Thread Mark Hamstra
r second in Spark. So, >> yes, >> web speed SQL queries are definitely possible. Read my new blog post: >> >> http://velvia.github.io/Spark-Concurrent-Fast-Queries/ >> >> and feel free to email me (at vel...@gmail.com) if you would like to >> follow >>

Re: Can we use spark inside a web service?

2016-03-10 Thread Tristan Nixon
gt; > http://velvia.github.io/Spark-Concurrent-Fast-Queries/ > > and feel free to email me (at vel...@gmail.com) if you would like to follow > up. > > -Evan > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabbl

Re: Can we use spark inside a web service?

2016-03-10 Thread Chris Fregly
Read my new blog post: > > http://velvia.github.io/Spark-Concurrent-Fast-Queries/ > > and feel free to email me (at vel...@gmail.com) if you would like to > follow > up. > > -Evan > > > > > -- > View this message in context: > http://apache-spark-

Re: Can we use spark inside a web service?

2016-03-10 Thread velvia.github
free to email me (at vel...@gmail.com) if you would like to follow up. -Evan -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Can-we-use-spark-inside-a-web-service-tp26426p26451.html Sent from the Apache Spark User List mailing list archive at Nabble.com