Re: Using kafka in non million users environment

2014-08-20 Thread Justin Maltat
Thanks Philip and Anand for the hints. I fill more comfortable going further now. On Wed, Aug 20, 2014 at 6:49 PM, Anand Nalya wrote: > For operating kafka across multiple data centers have a look at > https://kafka.apache.org/08/ops.html and MirrorMaker ( > https://kafka.apache.org/08/tools.h

Re: Using kafka in non million users environment

2014-08-20 Thread Anand Nalya
For operating kafka across multiple data centers have a look at https://kafka.apache.org/08/ops.html and MirrorMaker ( https://kafka.apache.org/08/tools.html) On 20 August 2014 04:09, Justin Maltat wrote: > Hi, > > As of today, our company IT is mainly composed of domain specific > software (pr

Re: Using kafka in non million users environment

2014-08-19 Thread Philip O'Toole
>>- we have a low data traffic compared to your figures: around 30 GB a day. Will it be an issue? I have personal experience that Kafka deals extremely well with very low-volumes, as well as very high. I have used Kafka for small integration-test setups, as well as large production systems. Kaf

Re: Using kafka in non million users environment

2014-08-19 Thread Justin Maltat
Hi, As of today, our company IT is mainly composed of domain specific software (proprietary and homemade). We could like to migrate them one after another to a microservice architecture with Kafka as the data pipeline. With the system now in place it's quite difficult to have a common data flow be

Re: Using kafka in non million users environment

2014-08-19 Thread Daniel Compton
Hi Justin It sounds like Kafka could be a good fit for your environment. Are you able to tell us more about the kinds of applications you will be running? Daniel. > On 19/08/2014, at 10:53 am, Justin Maltat wrote: > > Hello, > > I'm managing a study to explore possibilities for migrating a m