Re: Is kafka suitable for our architecture?

2014-10-10 Thread Albert Vila
r to stick with only one technology to keep everything as simple as possible. > Christian > Thanks > > On Thu, Oct 9, 2014 at 11:57 PM, Albert Vila > wrote: > > > Hi > > > > We process data in real time, and we are taking a look at Storm and Spark > > st

Re: Is kafka suitable for our architecture?

2014-10-09 Thread Albert Vila
switch in order > to > > optimize intra-cluster communications when moving data is unavoidable. > > > > -Will > > > > On Thu, Oct 9, 2014 at 7:57 AM, Albert Vila > wrote: > > > >> Hi > >> > >> I just came across Kafta when I was

Is kafka suitable for our architecture?

2014-10-09 Thread Albert Vila
Hi I just came across Kafta when I was trying to find solutions to scale our current architecture. We are currently downloading and processing 6M documents per day from online and social media. We have a different workflow for each type of document, but some of the steps are keyword extraction, l