Re: Discussion on requirements for Data Encryption functionality in Kafka (KIP-317)

2018-10-02 Thread mikegray831
Hi Sönke, I would be very interested in participating in this conversation. Very interested in how TDE might work in Kafka! I’m coming with several colleagues and will see if they’re interested in participating as well. Thanks, Mike Grayson On 2018/10/02 11:19:36, Sönke Liebau wrote: >

Re: Global/Restore consumer use auto.offset.reset = none vs. OffsetOutOfRangeException

2018-10-02 Thread Matthias J. Sax
It is by design to set the reset policy to "none" (https://issues.apache.org/jira/browse/KAFKA-6121), and not allowed by design to overwrite this (there might be a workaround for you though). However, Streams should not die but catch the exception and recover from it automatically. What version

KafkaBroker's auto thread dumps every 15mins

2018-10-02 Thread Vanteru, Rammohan Reddy
Hi Kafka Users, We have 3 KafkaBrokers(i.e kafka version 0.11) configured which is auto dumping thread dumps for every 12mins without posting any errors or warnings in logs but brokers are not crashing. At the moment Kafka brokers not experiencing any high load. Let me know if anyone has faced

Getting topic offset for given timestamp | GetOffsetShell is not working|KAFKA-1.1.1

2018-10-02 Thread dhiraj sardana
Hello All, I wanted to check offset for given timestamp via *shell command*, so i checked 'GetOffsetShell' for that purpose, but its not working properly. bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic test_kafka --time *1538470622047* test_kafka:0:

Global/Restore consumer use auto.offset.reset = none vs. OffsetOutOfRangeException

2018-10-02 Thread Patrik Kleindl
Hi We had several incidents where a streams application crashed while maintaining a global state store. Updating global state failed. You can restart KafkaStreams to recover from this error.: org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets out of range with no configured

Discussion on requirements for Data Encryption functionality in Kafka (KIP-317)

2018-10-02 Thread Sönke Liebau
Hi all, I have created KIP-317 [1] a while ago, which outlines an implementation proposal to add transparent data encryption functionality to Kafka. The KIP in its current form is somewhat rigid in its implementation, I will rework this to become extensible over the next few days to allow for

Re: [ANNOUNCE] New committer: Colin McCabe

2018-10-02 Thread James Cheng
Congrats, Colin! -James > On Sep 25, 2018, at 1:39 AM, Ismael Juma wrote: > > Hi all, > > The PMC for Apache Kafka has invited Colin McCabe as a committer and we are > pleased to announce that he has accepted! > > Colin has contributed 101 commits and 8 KIPs including significant >