Re: Idempotent Producers and Exactly Once Consumers

2019-09-29 Thread christopher palm
wrote: > > You can achieve exactly once on a consumer by enabling read committed and > > manually committing the offset as soon as you receive a message. That way > > you know that at next poll you won't get old message again. > > > > On Fri, Sep 27, 2019, 6:24 AM christo

Re: Idempotent Producers and Exactly Once Consumers

2019-09-27 Thread christopher palm
I had a similar question, and just watched the video on the confluent.io site about this. >From what I understand idempotence and transactions are there to solve the duplicate writes and exactly once processing, respectively. Is what you are stating below is that this only works if we produce

Re: KafkaProducer Retries in .9.0.1

2016-04-21 Thread christopher palm
g to take a look if someone else has filed a related issue about it. > > Regards, > Nicolas PHUNG > > On Thu, Apr 7, 2016 at 5:15 AM, christopher palm <cpa...@gmail.com> wrote: > > > Hi Thanks for the suggestion. > > I lowered the broker message.ma

KafkaProducer Retries in .9.0.1

2016-04-05 Thread christopher palm
Hi All, I am working with the KafkaProducer using the properties below, so that the producer keeps trying to send upon failure on Kafka .9.0.1. I am forcing a failure by setting my buffersize smaller than my payload,which causes the expected exception below. I don't see the producer retry to

Re: Security with SSL and not Kerberos?

2016-03-22 Thread christopher palm
> --consumer --topic securing-kafka --group securing-kafka-group > > Enabling the authorizer log is a good way to figure out the principal if > you don't know it. > > Hope this helps, > Ismael > > On Mon, Mar 21, 2016 at 10:27 PM, Raghavan, Gopal <gopal.ragha...@here.com >

Security with SSL and not Kerberos?

2016-03-21 Thread christopher palm
Hi All, Does Kafka support SSL authentication and ACL authorization without Kerberos? If so, can different clients have their own SSL certificate on the same broker? In reading the following security article, it seems that Kerberos is an option but not required if SSL is used. Thanks, Chris

Re: MultiThreaded HLConsumer Exits before events are all consumed

2015-04-30 Thread christopher palm
AM, christopher palm cpa...@gmail.com wrote: Commenting out Example shutdown did not seem to make a difference, I added the print statement below to highlight the fact. The other threads still shut down, and only one thread lives on, eventually that dies after a few minutes as well

Re: MultiThreaded HLConsumer Exits before events are all consumed

2015-04-29 Thread christopher palm
connections to Kafka. remove this line the consumer threads will run forever On Wed, Apr 29, 2015 at 9:42 PM, christopher palm cpa...@gmail.com wrote: Hi All, I am trying to get a multi threaded HL consumer working against a 2 broker Kafka cluster with a 4 partition 2 replica topic

MultiThreaded HLConsumer Exits before events are all consumed

2015-04-29 Thread christopher palm
Hi All, I am trying to get a multi threaded HL consumer working against a 2 broker Kafka cluster with a 4 partition 2 replica topic. The consumer code is set to run with 4 threads, one for each partition. The producer code uses the default partitioner and loops indefinitely feeding events into