Re: Pros and cons of dockerizing kafka brokers?

2016-07-08 Thread Krish
, Jul 7, 2016 at 2:30 AM, Krish <krishnan.k.i...@gmail.com> wrote: > >> Hi, >> I am currently testing a custom docker volume driver plugin for AWS >> EFS/EBS >> access and mounting. So, running kafka broker inside a container makes >> will >> ease up

Pros and cons of dockerizing kafka brokers?

2016-07-07 Thread Krish
, since kafka forms the base of our setup, I can think of making is use the host networking stack, and increase ulimits for the container. I would like to know if and when kafka becomes greedy and cannibalizes resources; I can also ensure that it runs on a dedicated machine. Thanks. Best, Krish

Re: Expired messages in kafka topic

2016-07-07 Thread Krish
inters to resolved this please. -- κρισhναν On Thu, Jun 23, 2016 at 11:06 PM, Krish <krishnan.k.i...@gmail.com> wrote: > Gwen, > Have selected priority 'minor', component as 'core', have assigned no > labels. > Jira link: https://issues.apache.org/jira/browse/KAFKA-3895. > >

Re: Expired messages in kafka topic

2016-06-23 Thread Krish
Christian Posta > <christian.po...@gmail.com> wrote: > > Sounds like something a traditional message broker (ie, ActiveMQ) would > be > > able to do with a TTL setting and expiry. Expired messages get moved to a > > DLQ. > > > > On Thu, Jun 23, 2

Re: Expired messages in kafka topic

2016-06-23 Thread Krish
y it means that a dead-letter Q like functionality can get multiple messages, and anyone draining the queue needs to take care of duplicates. Right so far? -- κρισhναν On Thu, Jun 23, 2016 at 9:21 PM, Krish <krishnan.k.i...@gmail.com> wrote: > Well, we are already using Kafka an

Re: Expired messages in kafka topic

2016-06-23 Thread Krish
, 2016 at 9:16 PM, Christian Posta <christian.po...@gmail.com> wrote: > Sounds like something a traditional message broker (ie, ActiveMQ) would be > able to do with a TTL setting and expiry. Expired messages get moved to a > DLQ. > > On Thu, Jun 23, 2016 at 2:45 AM, Krish <k

Re: Expired messages in kafka topic

2016-06-23 Thread Krish
> > Thanks > > Tom Crayford > Heroku Kafka > > On Thu, Jun 23, 2016 at 10:45 AM, Krish <krishnan.k.i...@gmail.com> wrote: > >> Hi, >> I am trying to design a real-time application where message timeout can be >> as low as a minute or two (messa

Expired messages in kafka topic

2016-06-23 Thread Krish
would like to know if a message has expired and then park it in some topic till as such time that a service can dequeue, process it and/or investigate it. Thanks. Best, Krish